[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:37:46 dist-test-slave-szdr krb5kdc[5617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:37:46 dist-test-slave-szdr krb5kdc[5617](info): set up 2 sockets
Jun 20 11:37:46 dist-test-slave-szdr krb5kdc[5617](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 11:37:47.275043 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:37:47 dist-test-slave-szdr krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:37:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:37:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:37:47.402118 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:34411
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:34411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:47.487215 5630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:47.487452 5630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:47.487475 5630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:47.488691 5630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:37:47.488732 5630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:47.488746 5630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:47.488759 5630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:37:47.488772 5630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:37:47.490303 5630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:34411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:34411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:47.490525 5630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:47.490720 5630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:47.495814 5636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:47.495863 5630 server_base.cc:1034] running on GCE node
W20260620 11:37:47.495822 5638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:47.495838 5635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:47.496296 5630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:47.496570 5630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:47.497725 5630 hybrid_clock.cc:648] HybridClock initialized: now 1781955467497699 us; error 41 us; skew 500 ppm
Jun 20 11:37:47 dist-test-slave-szdr krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955467, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:47.500372 5630 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:37:47.500931 5630 webserver.cc:460] Webserver started at http://127.5.122.190:44507/ using document root <none> and password file <none>
I20260620 11:37:47.501174 5630 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:47.501251 5630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:47.501403 5630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:47.502597 5630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "cc631273e6ec4159a133e92cca33e742"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "7dec0c43a24e1c7d9bb7c1760c40d8c0"
server_key_iv: "24ed9d5d3e8c3bdb5df40f197a3ca8af"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.503000 5630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "cc631273e6ec4159a133e92cca33e742"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "7dec0c43a24e1c7d9bb7c1760c40d8c0"
server_key_iv: "24ed9d5d3e8c3bdb5df40f197a3ca8af"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.504424 5630 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:37:47.505311 5645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:47.505505 5630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:47.505681 5630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "cc631273e6ec4159a133e92cca33e742"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "7dec0c43a24e1c7d9bb7c1760c40d8c0"
server_key_iv: "24ed9d5d3e8c3bdb5df40f197a3ca8af"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.505767 5630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:47.519026 5630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:47.521179 5630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:47.521360 5630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:47.525741 5630 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:34411
I20260620 11:37:47.525776 5697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:34411 every 8 connection(s)
I20260620 11:37:47.526201 5630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:37:47.527110 5698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:47.527573 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 5630
I20260620 11:37:47.527763 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:37:47.528460 5610 external_mini_cluster.cc:1428] Setting key 57c6266988643657b19deb5c266af2ea
I20260620 11:37:47.530141 5698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742: Bootstrap starting.
I20260620 11:37:47.530726 5698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:47.531010 5698 log.cc:824] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:47.531709 5698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742: No bootstrap required, opened a new log
I20260620 11:37:47.533542 5698 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc631273e6ec4159a133e92cca33e742" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34411 } }
I20260620 11:37:47.533711 5698 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:47.533783 5698 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc631273e6ec4159a133e92cca33e742, State: Initialized, Role: FOLLOWER
I20260620 11:37:47.533958 5698 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc631273e6ec4159a133e92cca33e742" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34411 } }
I20260620 11:37:47.534088 5698 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:37:47.534134 5698 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:37:47.534185 5698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:47.534737 5698 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc631273e6ec4159a133e92cca33e742" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34411 } }
I20260620 11:37:47.534858 5698 leader_election.cc:302] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc631273e6ec4159a133e92cca33e742; no voters:
I20260620 11:37:47.535040 5698 leader_election.cc:288] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:37:47.535090 5703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:47.535236 5703 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [term 1 LEADER]: Becoming Leader. State: Replica: cc631273e6ec4159a133e92cca33e742, State: Running, Role: LEADER
I20260620 11:37:47.535317 5698 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:37:47.535390 5703 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc631273e6ec4159a133e92cca33e742" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34411 } }
I20260620 11:37:47.535766 5703 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc631273e6ec4159a133e92cca33e742. Latest consensus state: current_term: 1 leader_uuid: "cc631273e6ec4159a133e92cca33e742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc631273e6ec4159a133e92cca33e742" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34411 } } }
I20260620 11:37:47.535909 5703 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:47.536378 5712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:37:47.536859 5705 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc631273e6ec4159a133e92cca33e742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc631273e6ec4159a133e92cca33e742" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34411 } } }
I20260620 11:37:47.536936 5705 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742 [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:47.536952 5712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:37:47.538828 5712 catalog_manager.cc:1347] Generated new cluster ID: 302de13370b6481fac2115a1b4897c5f
I20260620 11:37:47.538926 5712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:37:47 dist-test-slave-szdr krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:47.547021 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:47.529423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.5.122.190:34411)
0620 11:37:47.529667 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:47.529677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:47.529883 (+ 206us) server_negotiation.cc:407] Connection header received
0620 11:37:47.532157 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:47.532165 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:47.532213 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:47.532367 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:47.534051 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.535642 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.537346 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.537906 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.538093 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:47.542742 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:47.542746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:47.542746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:47.542759 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:47.544798 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:47.545334 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:47.545336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:47.545337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:47.545437 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:47.545735 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:47.545736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:47.545736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:47.546066 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:47.546224 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:47.546645 (+ 421us) server_negotiation.cc:299] Negotiation successful
0620 11:37:47.546792 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260620 11:37:47.571571 5712 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:37:47.572202 5712 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:37:47.583468 5712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc631273e6ec4159a133e92cca33e742: Generated new TSK 0
I20260620 11:37:47.583710 5712 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:37:47.617753 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:34411
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:47.701395 5724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:47.701623 5724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:47.701650 5724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:47.702955 5724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:47.703015 5724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:47.703085 5724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:37:47.704682 5724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:34411
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:47.704970 5724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:47.705231 5724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:37:47.710623 5724 server_base.cc:1034] running on GCE node
W20260620 11:37:47.710590 5732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:47.710633 5730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:47.710593 5729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:47.711061 5724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:47.711360 5724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:47.712517 5724 hybrid_clock.cc:648] HybridClock initialized: now 1781955467712495 us; error 41 us; skew 500 ppm
Jun 20 11:37:47 dist-test-slave-szdr krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955467, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:47.715366 5724 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:37:47.715912 5724 webserver.cc:460] Webserver started at http://127.5.122.129:34445/ using document root <none> and password file <none>
I20260620 11:37:47.716182 5724 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:47.716231 5724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:47.716388 5724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:47.717527 5724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "211c59233e284bd0b1e92d2bf61af9cb"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "0f57919a9a61bc62c51c91a81cd74381"
server_key_iv: "cd1f5da7fd54f96685c378c29dce2100"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.717926 5724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "211c59233e284bd0b1e92d2bf61af9cb"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "0f57919a9a61bc62c51c91a81cd74381"
server_key_iv: "cd1f5da7fd54f96685c378c29dce2100"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.719451 5724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:37:47.720279 5739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:47.720500 5724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:37:47.720614 5724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "211c59233e284bd0b1e92d2bf61af9cb"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "0f57919a9a61bc62c51c91a81cd74381"
server_key_iv: "cd1f5da7fd54f96685c378c29dce2100"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.720700 5724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:47.727560 5724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:47.729537 5724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:47.729714 5724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:47.730017 5724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:37:47.730397 5724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:47.730434 5724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:47.730458 5724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:47.730507 5724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:47.736783 5724 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:33179
I20260620 11:37:47.736825 5852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:33179 every 8 connection(s)
I20260620 11:37:47.737166 5724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:37:47.742975 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 5724
I20260620 11:37:47.743057 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
Jun 20 11:37:47 dist-test-slave-szdr krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955467, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:47.743230 5610 external_mini_cluster.cc:1428] Setting key 257dbbb0b04b9648ef36bb8236fd69ab
I20260620 11:37:47.746984 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:47.737868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:43105 (local address 127.5.122.190:34411)
0620 11:37:47.737939 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:47.737942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:47.738215 (+ 273us) server_negotiation.cc:407] Connection header received
0620 11:37:47.738738 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:47.738740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:47.738750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:47.738809 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:47.739760 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.740248 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.741213 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.741400 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.741449 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:47.743805 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:47.743806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:47.743807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:47.743813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:47.745444 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:47.746022 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:47.746023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:47.746024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:47.746077 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:47.746366 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:47.746367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:47.746367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:47.746466 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:47.746512 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:47.746856 (+ 344us) server_negotiation.cc:299] Negotiation successful
0620 11:37:47.746920 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:37:47.747267 5855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:47.737852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:34411 (local address 127.5.122.129:43105)
0620 11:37:47.738101 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:47.738140 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:47.738616 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:47.738889 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:47.738893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:47.739474 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:47.739676 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:47.739680 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.740329 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.740330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:47.741140 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:47.741142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.741468 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.741468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:47.741564 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:47.742165 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:47.742174 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:47.743714 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:47.745536 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:47.745543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:47.745545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:47.745907 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:47.746145 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:47.746146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:47.746146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:47.746293 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:47.746529 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:47.746534 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:47.746711 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 11:37:47.746819 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 11:37:47.746997 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":143,"threads_started":1}
I20260620 11:37:47.747794 5853 heartbeater.cc:347] Connected to a master server at 127.5.122.190:34411
I20260620 11:37:47.747889 5853 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:47.748070 5853 heartbeater.cc:511] Master 127.5.122.190:34411 requested a full tablet report, sending...
I20260620 11:37:47.748600 5662 ts_manager.cc:194] Registered new tserver with Master: 211c59233e284bd0b1e92d2bf61af9cb (127.5.122.129:33179)
I20260620 11:37:47.749594 5662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:43105
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:37:47.769464 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:34411
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:47.852804 5858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:47.853093 5858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:47.853139 5858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:47.854426 5858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:47.854511 5858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:47.854557 5858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:37:47.856156 5858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:34411
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:47.856484 5858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:47.856743 5858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:47.862437 5866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:47.862426 5864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:47.862428 5863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:47.862524 5858 server_base.cc:1034] running on GCE node
I20260620 11:37:47.863024 5858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:47.863312 5858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:47.864486 5858 hybrid_clock.cc:648] HybridClock initialized: now 1781955467864481 us; error 40 us; skew 500 ppm
Jun 20 11:37:47 dist-test-slave-szdr krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955467, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:47.867292 5858 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:37:47.867841 5858 webserver.cc:460] Webserver started at http://127.5.122.130:37567/ using document root <none> and password file <none>
I20260620 11:37:47.868098 5858 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:47.868180 5858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:47.868367 5858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:47.869495 5858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "1b25aa236cb04ca9a823aacded53c122"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "e33e7e21de0983c2cd668deee6d1a9f0"
server_key_iv: "c7cbcc08bd14943ec1670a2dee16ee4d"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.869885 5858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "1b25aa236cb04ca9a823aacded53c122"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "e33e7e21de0983c2cd668deee6d1a9f0"
server_key_iv: "c7cbcc08bd14943ec1670a2dee16ee4d"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.871297 5858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:37:47.872152 5873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:47.872373 5858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:37:47.872480 5858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "1b25aa236cb04ca9a823aacded53c122"
format_stamp: "Formatted at 2026-06-20 11:37:47 on dist-test-slave-szdr"
server_key: "e33e7e21de0983c2cd668deee6d1a9f0"
server_key_iv: "c7cbcc08bd14943ec1670a2dee16ee4d"
server_key_version: "encryptionkey@0"
I20260620 11:37:47.872565 5858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:47.888775 5858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:47.890863 5858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:47.891045 5858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:47.891291 5858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:37:47.891629 5858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:47.891685 5858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:47.891719 5858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:47.891750 5858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:47.898033 5858 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:44925
I20260620 11:37:47.898108 5986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:44925 every 8 connection(s)
I20260620 11:37:47.898502 5858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:37:47.904875 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 5858
Jun 20 11:37:47 dist-test-slave-szdr krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955467, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:47.904999 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:37:47.905157 5610 external_mini_cluster.cc:1428] Setting key c914540bf423a9e8e74ca7c4ccfb83da
I20260620 11:37:47.908746 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:47.899345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:39635 (local address 127.5.122.190:34411)
0620 11:37:47.899406 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:47.899408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:47.899634 (+ 226us) server_negotiation.cc:407] Connection header received
0620 11:37:47.900200 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:47.900201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:47.900213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:47.900261 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:47.901303 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.901880 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.902949 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.903165 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.903258 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:47.905571 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:47.905572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:47.905572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:47.905580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:47.907212 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:47.907662 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:47.907664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:47.907664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:47.907732 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:47.908023 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:47.908025 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:47.908025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:47.908152 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:47.908232 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:47.908557 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 11:37:47.908649 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:37:47.908967 5989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:47.899319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:34411 (local address 127.5.122.130:39635)
0620 11:37:47.899503 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:47.899542 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:47.900078 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:47.900341 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:47.900345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:47.900956 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:47.901184 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:47.901189 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.902005 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.902006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:47.902848 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:47.902850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:47.903269 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:47.903271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:47.903414 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:47.903934 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:47.903943 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:47.905467 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:47.907320 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:47.907329 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:47.907332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:47.907578 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:47.907800 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:47.907801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:47.907802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:47.907965 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:47.908233 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:47.908241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:47.908395 (+ 154us) client_negotiation.cc:769] Sending connection context
0620 11:37:47.908506 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 11:37:47.908709 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260620 11:37:47.909493 5987 heartbeater.cc:347] Connected to a master server at 127.5.122.190:34411
I20260620 11:37:47.909621 5987 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:47.909816 5987 heartbeater.cc:511] Master 127.5.122.190:34411 requested a full tablet report, sending...
I20260620 11:37:47.910284 5662 ts_manager.cc:194] Registered new tserver with Master: 1b25aa236cb04ca9a823aacded53c122 (127.5.122.130:44925)
I20260620 11:37:47.910880 5662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:39635
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:37:47.933506 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:34411
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:48.019080 5992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:48.019318 5992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:48.019343 5992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:48.020568 5992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:48.020612 5992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:48.020674 5992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:37:48.022269 5992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:34411
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:48.022522 5992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:48.022785 5992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:48.028113 5997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:48.028151 5998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:48.028359 5992 server_base.cc:1034] running on GCE node
W20260620 11:37:48.028373 6000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:48.028716 5992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:48.028919 5992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:48.030085 5992 hybrid_clock.cc:648] HybridClock initialized: now 1781955468030073 us; error 43 us; skew 500 ppm
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955468, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:48.032720 5992 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:37:48.033210 5992 webserver.cc:460] Webserver started at http://127.5.122.131:33841/ using document root <none> and password file <none>
I20260620 11:37:48.033504 5992 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:48.033598 5992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:48.033746 5992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:48.034885 5992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "7a259ade51374acaad478ef0f78f5853"
format_stamp: "Formatted at 2026-06-20 11:37:48 on dist-test-slave-szdr"
server_key: "b3918f013d2f549781a29a757d3f85a3"
server_key_iv: "94f8df1e2c16516127b388c7b8c21302"
server_key_version: "encryptionkey@0"
I20260620 11:37:48.035316 5992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "7a259ade51374acaad478ef0f78f5853"
format_stamp: "Formatted at 2026-06-20 11:37:48 on dist-test-slave-szdr"
server_key: "b3918f013d2f549781a29a757d3f85a3"
server_key_iv: "94f8df1e2c16516127b388c7b8c21302"
server_key_version: "encryptionkey@0"
I20260620 11:37:48.036854 5992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:48.037652 6007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:48.037832 5992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:48.037932 5992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "7a259ade51374acaad478ef0f78f5853"
format_stamp: "Formatted at 2026-06-20 11:37:48 on dist-test-slave-szdr"
server_key: "b3918f013d2f549781a29a757d3f85a3"
server_key_iv: "94f8df1e2c16516127b388c7b8c21302"
server_key_version: "encryptionkey@0"
I20260620 11:37:48.038031 5992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:48.046535 5992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:48.048588 5992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:48.048758 5992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:48.049002 5992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:37:48.049297 5992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:48.049328 5992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:48.049351 5992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:48.049363 5992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:48.055364 5992 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:39403
I20260620 11:37:48.055431 6120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:39403 every 8 connection(s)
I20260620 11:37:48.055761 5992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:37:48.059055 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 5992
I20260620 11:37:48.059182 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:37:48.059345 5610 external_mini_cluster.cc:1428] Setting key 99bba52b17057ebdab88b05f5715af89
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955468, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:48.065315 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:48.056538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:47653 (local address 127.5.122.190:34411)
0620 11:37:48.056584 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:48.056587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:48.056782 (+ 195us) server_negotiation.cc:407] Connection header received
0620 11:37:48.057275 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:48.057276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:48.057285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:48.057339 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:48.058490 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.058997 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.060384 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.060638 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.060704 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:48.062855 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:48.062857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:48.062857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:48.062864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:48.064153 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:48.064518 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:48.064518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:48.064519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:48.064567 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:48.064797 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:48.064797 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:48.064797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:48.064897 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:48.064948 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:48.065179 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 11:37:48.065244 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:37:48.065538 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:48.056491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:34411 (local address 127.5.122.131:47653)
0620 11:37:48.056651 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:48.056694 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:48.057191 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:48.057406 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:48.057410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:48.058089 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:48.058390 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:48.058398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.059111 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.059112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:48.060266 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:48.060269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.060730 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.060731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:48.060839 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:48.061306 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:48.061314 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:48.062767 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:48.064232 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:48.064237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:48.064239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:48.064454 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:48.064623 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:48.064623 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:48.064624 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:48.064743 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:48.064950 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:48.064954 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:48.065067 (+ 113us) client_negotiation.cc:769] Sending connection context
0620 11:37:48.065153 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 11:37:48.065280 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260620 11:37:48.066095 6121 heartbeater.cc:347] Connected to a master server at 127.5.122.190:34411
I20260620 11:37:48.066213 6121 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:48.066390 6121 heartbeater.cc:511] Master 127.5.122.190:34411 requested a full tablet report, sending...
I20260620 11:37:48.066841 5662 ts_manager.cc:194] Registered new tserver with Master: 7a259ade51374acaad478ef0f78f5853 (127.5.122.131:39403)
I20260620 11:37:48.067235 5662 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:47653
I20260620 11:37:48.071458 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955468, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955468, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:37:48.095010 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:48.087141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.5.122.129:33179)
0620 11:37:48.087314 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:48.087318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:48.087330 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:37:48.087357 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:48.087359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:48.087373 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:48.087442 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:48.088009 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.088773 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.089488 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.089812 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.089865 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:48.092106 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:48.092115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:48.092116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:48.092123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:48.093809 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:48.094222 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:48.094226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:48.094226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:48.094317 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:48.094479 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:48.094479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:48.094479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:48.094622 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:48.094729 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:48.094786 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 11:37:48.094857 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
W20260620 11:37:48.095419 5767 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35628
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[5617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[5617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:37:48.118739 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:48.111410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35630 (local address 127.5.122.129:33179)
0620 11:37:48.111480 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:48.111482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:48.111492 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:48.111566 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:48.111566 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:48.111577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:48.111629 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:48.112186 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.112696 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.113391 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:48.113588 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:48.113639 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:48.115978 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:48.115980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:48.115981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:48.115997 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:48.117452 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:48.117821 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:48.117823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:48.117823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:48.117883 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:48.118199 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:48.118200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:48.118200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:48.118313 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:48.118439 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:48.118539 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 11:37:48.118630 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:37:48.119374 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 5724
I20260620 11:37:48.124917 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 5858
I20260620 11:37:48.129215 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 5992
I20260620 11:37:48.133466 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 5630
2026-06-20T11:37:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1992 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[6135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[6135](info): set up 2 sockets
Jun 20 11:37:48 dist-test-slave-szdr krb5kdc[6135](info): commencing operation
krb5kdc: starting...
W20260620 11:37:49.165721 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:37:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:37:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:37:49.288612 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:46495
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:46495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:49.371948 6148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:49.372177 6148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:49.372200 6148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:49.373520 6148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:37:49.373565 6148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:49.373580 6148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:49.373593 6148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:37:49.373605 6148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:37:49.375222 6148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:46495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:46495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:49.375505 6148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:49.375752 6148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:49.380867 6154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:49.380863 6153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:49.380973 6148 server_base.cc:1034] running on GCE node
W20260620 11:37:49.380863 6156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:49.381390 6148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:49.381712 6148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:49.382889 6148 hybrid_clock.cc:648] HybridClock initialized: now 1781955469382874 us; error 44 us; skew 500 ppm
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:49.385588 6148 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:37:49.386134 6148 webserver.cc:460] Webserver started at http://127.5.122.190:34977/ using document root <none> and password file <none>
I20260620 11:37:49.386391 6148 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:49.386461 6148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:49.386617 6148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:49.387739 6148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "2c5237030a3d49859566227fe9fda36b"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "4ed72e86a93654b739698b0b4cb7437f"
server_key_iv: "b53f5de59f4f7e3849498d66fffb03fa"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.388108 6148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "2c5237030a3d49859566227fe9fda36b"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "4ed72e86a93654b739698b0b4cb7437f"
server_key_iv: "b53f5de59f4f7e3849498d66fffb03fa"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.389474 6148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:49.390319 6163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.390478 6148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:49.390609 6148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "2c5237030a3d49859566227fe9fda36b"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "4ed72e86a93654b739698b0b4cb7437f"
server_key_iv: "b53f5de59f4f7e3849498d66fffb03fa"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.390690 6148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:49.416666 6148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:49.419382 6148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:49.419569 6148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:49.423607 6148 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:46495
I20260620 11:37:49.423641 6215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:46495 every 8 connection(s)
I20260620 11:37:49.424046 6148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:37:49.424898 6216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:49.427284 6216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b: Bootstrap starting.
I20260620 11:37:49.427845 6216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:49.428110 6216 log.cc:824] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:49.428812 6216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b: No bootstrap required, opened a new log
I20260620 11:37:49.430500 6216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5237030a3d49859566227fe9fda36b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46495 } }
I20260620 11:37:49.430658 6216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:49.430682 6216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c5237030a3d49859566227fe9fda36b, State: Initialized, Role: FOLLOWER
I20260620 11:37:49.430832 6216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5237030a3d49859566227fe9fda36b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46495 } }
I20260620 11:37:49.430912 6216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:37:49.430935 6216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:37:49.430964 6216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:49.431473 6216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5237030a3d49859566227fe9fda36b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46495 } }
I20260620 11:37:49.431563 6216 leader_election.cc:302] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c5237030a3d49859566227fe9fda36b; no voters:
I20260620 11:37:49.431721 6216 leader_election.cc:288] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:37:49.431751 6219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:49.431872 6219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [term 1 LEADER]: Becoming Leader. State: Replica: 2c5237030a3d49859566227fe9fda36b, State: Running, Role: LEADER
I20260620 11:37:49.431957 6219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5237030a3d49859566227fe9fda36b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46495 } }
I20260620 11:37:49.431993 6216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:37:49.432466 6221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c5237030a3d49859566227fe9fda36b. Latest consensus state: current_term: 1 leader_uuid: "2c5237030a3d49859566227fe9fda36b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5237030a3d49859566227fe9fda36b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46495 } } }
I20260620 11:37:49.432565 6221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:49.432690 6220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c5237030a3d49859566227fe9fda36b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c5237030a3d49859566227fe9fda36b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46495 } } }
I20260620 11:37:49.432792 6224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:37:49.432829 6220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:49.433377 6224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:37:49.433938 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 6148
I20260620 11:37:49.434059 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:37:49.434235 5610 external_mini_cluster.cc:1428] Setting key 64fd04ac831c7e9d1343a121669d6955
I20260620 11:37:49.435359 6224 catalog_manager.cc:1347] Generated new cluster ID: 239f09d630914b459ea2e8632ddcf24c
I20260620 11:37:49.435420 6224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:49.446867 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.434751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.5.122.190:46495)
0620 11:37:49.434930 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.434940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.434990 (+ 50us) server_negotiation.cc:407] Connection header received
0620 11:37:49.435119 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.435128 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.435193 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.436119 (+ 926us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:49.436218 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.437679 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.438869 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.439522 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.439627 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.442729 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:49.442735 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:49.442735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:49.442751 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:49.444989 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.445385 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.445388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.445389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.445528 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.445726 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.445727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.445727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.446123 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:49.446307 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.446407 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.446607 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260620 11:37:49.454177 6224 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:37:49.454960 6224 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:37:49.462798 6224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c5237030a3d49859566227fe9fda36b: Generated new TSK 0
I20260620 11:37:49.463075 6224 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:37:49.497351 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46495
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:49.581075 6242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:49.581303 6242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:49.581331 6242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:49.582636 6242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:49.582695 6242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:49.582726 6242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:37:49.584275 6242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:49.584497 6242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:49.584692 6242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:37:49.590171 6242 server_base.cc:1034] running on GCE node
W20260620 11:37:49.590150 6248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:49.590150 6250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:49.590144 6247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:49.590559 6242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:49.590832 6242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:49.592023 6242 hybrid_clock.cc:648] HybridClock initialized: now 1781955469592019 us; error 41 us; skew 500 ppm
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:49.594659 6242 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:37:49.595155 6242 webserver.cc:460] Webserver started at http://127.5.122.129:38721/ using document root <none> and password file <none>
I20260620 11:37:49.595393 6242 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:49.595466 6242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:49.595626 6242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:49.596745 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "48cc49b9891341dbb966fe70cd7e1a09"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "78f1b01904e79223bdad85c441b0aba6"
server_key_iv: "a76e817287f0c0f5eae3f3aece6ddcdd"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.597131 6242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "48cc49b9891341dbb966fe70cd7e1a09"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "78f1b01904e79223bdad85c441b0aba6"
server_key_iv: "a76e817287f0c0f5eae3f3aece6ddcdd"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.598645 6242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:49.599470 6257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.599655 6242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:49.599767 6242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "48cc49b9891341dbb966fe70cd7e1a09"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "78f1b01904e79223bdad85c441b0aba6"
server_key_iv: "a76e817287f0c0f5eae3f3aece6ddcdd"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.599849 6242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:49.608505 6242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:49.610563 6242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:49.610745 6242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:49.611011 6242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:37:49.611406 6242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:49.611464 6242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.611501 6242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:49.611547 6242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.617506 6242 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:43631
I20260620 11:37:49.617553 6370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:43631 every 8 connection(s)
I20260620 11:37:49.617918 6242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:37:49.622529 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 6242
I20260620 11:37:49.622643 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:37:49.622815 5610 external_mini_cluster.cc:1428] Setting key 52db9a332ecdb8099787afee6b9a818c
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:49.628986 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.618735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:46495 (local address 127.5.122.129:51873)
0620 11:37:49.618919 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:49.618958 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:49.619906 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:49.620177 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:49.620180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:49.620451 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:49.620687 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.620692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.621406 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.621407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.622449 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.622453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.623054 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.623055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.623173 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.623804 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:49.623815 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:49.625551 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:49.627512 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:49.627525 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:49.627528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:49.627772 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:49.628003 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:49.628004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:49.628004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:49.628126 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:49.628446 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:49.628450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:49.628589 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 11:37:49.628677 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 11:37:49.628811 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
I20260620 11:37:49.628974 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.618774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:51873 (local address 127.5.122.190:46495)
0620 11:37:49.618860 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.618863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.619048 (+ 185us) server_negotiation.cc:407] Connection header received
0620 11:37:49.620009 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.620010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.620024 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.620084 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:49.620791 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.621316 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.622568 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.622865 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.622949 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.625662 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:49.625664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:49.625664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:49.625672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:49.627413 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.627852 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.627855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.627855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.627923 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.628193 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.628194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.628195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.628374 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:49.628455 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.628730 (+ 275us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.628833 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 11:37:49.629662 6371 heartbeater.cc:347] Connected to a master server at 127.5.122.190:46495
I20260620 11:37:49.629786 6371 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:49.630040 6371 heartbeater.cc:511] Master 127.5.122.190:46495 requested a full tablet report, sending...
I20260620 11:37:49.630620 6180 ts_manager.cc:194] Registered new tserver with Master: 48cc49b9891341dbb966fe70cd7e1a09 (127.5.122.129:43631)
I20260620 11:37:49.631268 6180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:51873
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:37:49.650895 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46495
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:49.736248 6376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:49.736492 6376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:49.736522 6376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:49.737924 6376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:49.738036 6376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:49.738095 6376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:37:49.739779 6376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:49.740095 6376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:49.740355 6376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:49.745707 6384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:49.745716 6381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:49.745715 6382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:49.745988 6376 server_base.cc:1034] running on GCE node
I20260620 11:37:49.746212 6376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:49.746430 6376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:49.747576 6376 hybrid_clock.cc:648] HybridClock initialized: now 1781955469747562 us; error 29 us; skew 500 ppm
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:49.750348 6376 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:37:49.750864 6376 webserver.cc:460] Webserver started at http://127.5.122.130:45681/ using document root <none> and password file <none>
I20260620 11:37:49.751081 6376 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:49.751155 6376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:49.751262 6376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:49.752384 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "1d4978558e594f60b960935be208a531"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "a4bbfee1935077fcdf8ad21fa5b38eb4"
server_key_iv: "66bf27cae0eb5f72da1980726bf270f2"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.752722 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "1d4978558e594f60b960935be208a531"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "a4bbfee1935077fcdf8ad21fa5b38eb4"
server_key_iv: "66bf27cae0eb5f72da1980726bf270f2"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.754117 6376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:49.754876 6391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.755057 6376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:49.755175 6376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "1d4978558e594f60b960935be208a531"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "a4bbfee1935077fcdf8ad21fa5b38eb4"
server_key_iv: "66bf27cae0eb5f72da1980726bf270f2"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.755235 6376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:49.779146 6376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:49.781175 6376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:49.781296 6376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:49.781503 6376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:37:49.781839 6376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:49.781874 6376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.781898 6376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:49.781945 6376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.788200 6376 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:37617
I20260620 11:37:49.788241 6504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:37617 every 8 connection(s)
I20260620 11:37:49.788610 6376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:49.796352 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 6376
I20260620 11:37:49.796469 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:37:49.796674 5610 external_mini_cluster.cc:1428] Setting key 8e91d4cbb97a5dd6f5a0f8358f99a49e
I20260620 11:37:49.798909 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.789392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:52391 (local address 127.5.122.190:46495)
0620 11:37:49.789475 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.789477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.789662 (+ 185us) server_negotiation.cc:407] Connection header received
0620 11:37:49.790731 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.790732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.790743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.790800 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:49.791455 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.791964 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.792977 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.793168 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.793255 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.795650 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:49.795651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:49.795652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:49.795661 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:49.797318 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.797752 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.797753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.797754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.797822 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.798138 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.798139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.798140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.798301 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:49.798375 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.798671 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.798777 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:37:49.799214 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.789356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:46495 (local address 127.5.122.130:52391)
0620 11:37:49.789527 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:49.789566 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:49.790627 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:49.790862 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:49.790866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:49.791160 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:49.791370 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.791375 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.792058 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.792059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.792902 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.792904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.793276 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.793278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.793417 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.794084 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:49.794094 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:49.795540 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:49.797403 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:49.797411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:49.797414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:49.797674 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:49.797892 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:49.797892 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:49.797893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:49.798064 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:49.798368 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:49.798372 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:49.798529 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 11:37:49.798612 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 11:37:49.798747 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260620 11:37:49.799815 6505 heartbeater.cc:347] Connected to a master server at 127.5.122.190:46495
I20260620 11:37:49.799937 6505 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:49.800150 6505 heartbeater.cc:511] Master 127.5.122.190:46495 requested a full tablet report, sending...
I20260620 11:37:49.800590 6180 ts_manager.cc:194] Registered new tserver with Master: 1d4978558e594f60b960935be208a531 (127.5.122.130:37617)
I20260620 11:37:49.800993 6180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:52391
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:37:49.825686 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46495
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:49.909178 6510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:49.909446 6510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:49.909488 6510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:49.910802 6510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:49.910874 6510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:49.910918 6510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:37:49.912484 6510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46495
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:49.912775 6510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:49.913026 6510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:49.918318 6516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:49.918390 6518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:49.918390 6510 server_base.cc:1034] running on GCE node
W20260620 11:37:49.918322 6515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:49.918740 6510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:49.919025 6510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:49.920185 6510 hybrid_clock.cc:648] HybridClock initialized: now 1781955469920172 us; error 30 us; skew 500 ppm
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:49.922803 6510 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:37:49.923280 6510 webserver.cc:460] Webserver started at http://127.5.122.131:41825/ using document root <none> and password file <none>
I20260620 11:37:49.923523 6510 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:49.923570 6510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:49.923743 6510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:49.924827 6510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "fc8d1c561e2048e882688aca513314df"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "c82db73e86dcfdb671ecd32b986f2634"
server_key_iv: "da18d5f37920188f509f67c0c98208a6"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.925197 6510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "fc8d1c561e2048e882688aca513314df"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "c82db73e86dcfdb671ecd32b986f2634"
server_key_iv: "da18d5f37920188f509f67c0c98208a6"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.926743 6510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260620 11:37:49.927628 6525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.927834 6510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:49.927970 6510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "fc8d1c561e2048e882688aca513314df"
format_stamp: "Formatted at 2026-06-20 11:37:49 on dist-test-slave-szdr"
server_key: "c82db73e86dcfdb671ecd32b986f2634"
server_key_iv: "da18d5f37920188f509f67c0c98208a6"
server_key_version: "encryptionkey@0"
I20260620 11:37:49.928061 6510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:49.943773 6510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:49.946095 6510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:49.946277 6510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:49.946552 6510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:37:49.947001 6510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:49.947080 6510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.947132 6510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:49.947163 6510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:49.953660 6510 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:44277
I20260620 11:37:49.953739 6638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:44277 every 8 connection(s)
I20260620 11:37:49.954188 6510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
Jun 20 11:37:49 dist-test-slave-szdr krb5kdc[6135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:49.960935 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 6510
I20260620 11:37:49.961066 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:37:49.961228 5610 external_mini_cluster.cc:1428] Setting key e2079d14acf6d79c5bc6f901b2450c1e
I20260620 11:37:49.964313 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.955007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:58577 (local address 127.5.122.190:46495)
0620 11:37:49.955098 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.955101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.955305 (+ 204us) server_negotiation.cc:407] Connection header received
0620 11:37:49.956332 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.956333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.956344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.956394 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:49.957089 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.957596 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.958786 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.958977 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.959038 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.961271 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:49.961273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:49.961273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:49.961281 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:49.962858 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.963315 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.963316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.963316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.963384 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.963653 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.963654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.963654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.963772 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:49.963842 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.964140 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.964220 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 11:37:49.964525 6641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.954961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:46495 (local address 127.5.122.131:58577)
0620 11:37:49.955162 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:49.955207 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:49.956228 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:49.956472 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:49.956477 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:49.956764 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:49.957005 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.957010 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.957735 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.957736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.958694 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.958697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.959061 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.959062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.959213 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.959710 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:49.959719 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:49.961163 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:49.962987 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:49.962994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:49.962996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:49.963236 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:49.963452 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:49.963453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:49.963454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:49.963579 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:49.963843 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:49.963850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:49.963998 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 11:37:49.964104 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 11:37:49.964270 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260620 11:37:49.964982 6639 heartbeater.cc:347] Connected to a master server at 127.5.122.190:46495
I20260620 11:37:49.965070 6639 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:49.965309 6639 heartbeater.cc:511] Master 127.5.122.190:46495 requested a full tablet report, sending...
I20260620 11:37:49.965740 6180 ts_manager.cc:194] Registered new tserver with Master: fc8d1c561e2048e882688aca513314df (127.5.122.131:44277)
I20260620 11:37:49.966267 6180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:58577
I20260620 11:37:49.973239 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:37:49.980736 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.974994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.5.122.190:46495)
0620 11:37:49.975062 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.975063 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.975135 (+ 72us) server_negotiation.cc:407] Connection header received
0620 11:37:49.975159 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.975159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.975172 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.975225 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:49.975737 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.976235 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.976910 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.977109 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.977168 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.978374 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:49.978375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:49.978376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:49.978383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:49.979709 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.980022 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.980023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.980023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.980074 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:49.980284 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:49.980285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:49.980285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:49.980429 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:49.980528 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.980605 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.980669 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:37:49.983170 6180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:37:49.984849 6180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:49.991659 6653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.987561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.5.122.131:44277)
0620 11:37:49.987756 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.987762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.987773 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:37:49.987907 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.987910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.987929 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.988012 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:49.988216 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.989126 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.990620 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.991249 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.991341 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.991404 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.991523 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.991566 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":89,"threads_started":1}
I20260620 11:37:49.991650 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.987487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:44277 (local address 127.0.0.1:44676)
0620 11:37:49.987664 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:49.987708 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:49.987777 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:49.988030 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:49.988034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:49.988036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:49.988117 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.988120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.989230 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.989232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.990526 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.990529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.991339 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.991340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.991418 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.991432 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 11:37:49.991507 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 11:37:49.991564 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":68,"threads_started":1}
I20260620 11:37:49.992050 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.987489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:43631 (local address 127.0.0.1:38138)
0620 11:37:49.987816 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:49.987826 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:49.987867 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:49.988156 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:49.988157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:49.988160 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:49.988216 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.988218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.989559 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.989560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.990632 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.990634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.991608 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.991609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.991674 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.991679 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:37:49.991913 (+ 234us) client_negotiation.cc:240] Negotiation successful
0620 11:37:49.991956 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"spinlock_wait_cycles":768,"thread_start_us":109,"threads_started":1}
I20260620 11:37:49.992129 6655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.987751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52426 (local address 127.5.122.130:37617)
0620 11:37:49.988255 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.988260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.988270 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:49.988307 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.988309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.988323 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.988409 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:49.988629 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.989673 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.990899 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.991642 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.991706 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.991757 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.992002 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.992044 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":76,"threads_started":1}
I20260620 11:37:49.992273 6652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.987508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:37617 (local address 127.0.0.1:52426)
0620 11:37:49.987937 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:49.987945 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:49.987995 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:49.988497 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:49.988498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:49.988500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:49.988536 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.988538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.989769 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.989770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.990813 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:49.990816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.991804 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.991805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:49.991877 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.991881 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:37:49.991928 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:37:49.991962 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260620 11:37:49.992365 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:49.987746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.5.122.129:43631)
0620 11:37:49.987923 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:49.987927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:49.987939 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:37:49.987999 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:49.988001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:49.988017 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:49.988106 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:49.988301 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.989437 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.990721 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:49.991443 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:49.991517 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:49.991572 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:49.992227 (+ 655us) server_negotiation.cc:299] Negotiation successful
0620 11:37:49.992275 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260620 11:37:49.992506 6573 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2ff92e4a84fb6b130e2b41e74c9a5 (DEFAULT_TABLE table=test-table [id=7b158ad3400c42ee974c9bc7441a6dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:49.992909 6573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2ff92e4a84fb6b130e2b41e74c9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:49.992986 6439 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2ff92e4a84fb6b130e2b41e74c9a5 (DEFAULT_TABLE table=test-table [id=7b158ad3400c42ee974c9bc7441a6dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:49.992977 6305 tablet_service.cc:1467] Processing CreateTablet for tablet 98a2ff92e4a84fb6b130e2b41e74c9a5 (DEFAULT_TABLE table=test-table [id=7b158ad3400c42ee974c9bc7441a6dd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:49.993316 6439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2ff92e4a84fb6b130e2b41e74c9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:49.993311 6305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98a2ff92e4a84fb6b130e2b41e74c9a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:49.995967 6656 tablet_bootstrap.cc:492] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df: Bootstrap starting.
I20260620 11:37:49.996706 6656 tablet_bootstrap.cc:654] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:49.996693 6657 tablet_bootstrap.cc:492] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09: Bootstrap starting.
I20260620 11:37:49.997026 6658 tablet_bootstrap.cc:492] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531: Bootstrap starting.
I20260620 11:37:49.997359 6656 log.cc:824] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:49.997402 6657 tablet_bootstrap.cc:654] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:49.997679 6658 tablet_bootstrap.cc:654] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:49.997960 6657 log.cc:824] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:49.998262 6658 log.cc:824] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:49.998353 6656 tablet_bootstrap.cc:492] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df: No bootstrap required, opened a new log
I20260620 11:37:49.998425 6656 ts_tablet_manager.cc:1397] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:37:49.998803 6657 tablet_bootstrap.cc:492] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09: No bootstrap required, opened a new log
I20260620 11:37:49.998876 6657 ts_tablet_manager.cc:1397] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:37:49.998983 6658 tablet_bootstrap.cc:492] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531: No bootstrap required, opened a new log
I20260620 11:37:49.999050 6658 ts_tablet_manager.cc:1397] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:37:50.000425 6658 raft_consensus.cc:348] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.000607 6658 raft_consensus.cc:374] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:50.000653 6658 raft_consensus.cc:729] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d4978558e594f60b960935be208a531, State: Initialized, Role: FOLLOWER
I20260620 11:37:50.000787 6658 consensus_queue.cc:260] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.000867 6656 raft_consensus.cc:348] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.000867 6657 raft_consensus.cc:348] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.001058 6657 raft_consensus.cc:374] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:50.001058 6656 raft_consensus.cc:374] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:50.001098 6656 raft_consensus.cc:729] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc8d1c561e2048e882688aca513314df, State: Initialized, Role: FOLLOWER
I20260620 11:37:50.001098 6657 raft_consensus.cc:729] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48cc49b9891341dbb966fe70cd7e1a09, State: Initialized, Role: FOLLOWER
I20260620 11:37:50.001127 6658 ts_tablet_manager.cc:1428] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:37:50.001258 6505 heartbeater.cc:503] Master 127.5.122.190:46495 was elected leader, sending a full tablet report...
I20260620 11:37:50.001235 6657 consensus_queue.cc:260] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.001236 6656 consensus_queue.cc:260] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.001518 6657 ts_tablet_manager.cc:1428] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 11:37:50.001519 6656 ts_tablet_manager.cc:1428] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:37:50.001582 6639 heartbeater.cc:503] Master 127.5.122.190:46495 was elected leader, sending a full tablet report...
I20260620 11:37:50.001658 6371 heartbeater.cc:503] Master 127.5.122.190:46495 was elected leader, sending a full tablet report...
W20260620 11:37:50.039153 6506 tablet.cc:2369] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:37:50.118711 6372 tablet.cc:2369] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:37:50.174728 6664 raft_consensus.cc:482] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:50.174916 6664 raft_consensus.cc:504] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.175300 6664 leader_election.cc:288] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d4978558e594f60b960935be208a531 (127.5.122.130:37617), fc8d1c561e2048e882688aca513314df (127.5.122.131:44277)
I20260620 11:37:50.178493 6655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:50.175609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:51399 (local address 127.5.122.130:37617)
0620 11:37:50.175667 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:50.175669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:50.175832 (+ 163us) server_negotiation.cc:407] Connection header received
0620 11:37:50.175908 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:50.175909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:50.175923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:50.176013 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:50.176187 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.176677 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.177652 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.178124 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.178220 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:50.178229 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:50.178382 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 11:37:50.178419 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 11:37:50.178521 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:50.175520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:37617 (local address 127.5.122.129:51399)
0620 11:37:50.175754 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:50.175767 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:50.175848 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:50.176018 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:50.176020 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:50.176021 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:50.176093 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:50.176102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.176761 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.176762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:50.177548 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:50.177551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.178219 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.178220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:50.178295 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:50.178305 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 11:37:50.178374 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 11:37:50.178419 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"spinlock_wait_cycles":14208,"thread_start_us":117,"threads_started":1}
I20260620 11:37:50.178853 6459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2ff92e4a84fb6b130e2b41e74c9a5" candidate_uuid: "48cc49b9891341dbb966fe70cd7e1a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4978558e594f60b960935be208a531" is_pre_election: true
I20260620 11:37:50.178903 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:50.175519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:44277 (local address 127.5.122.129:50823)
0620 11:37:50.175733 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:50.175746 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:50.175817 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:50.176005 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:50.176006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:50.176008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:50.176116 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:50.176118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.176824 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.176825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:50.177764 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:50.177766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.178484 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.178485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:50.178711 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:50.178717 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:37:50.178771 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:37:50.178813 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":118,"threads_started":1}
I20260620 11:37:50.178891 6653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:50.175707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:50823 (local address 127.5.122.131:44277)
0620 11:37:50.175783 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:50.175785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:50.175796 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:37:50.175908 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:50.175910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:50.175921 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:50.175999 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:50.176202 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.176721 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.177859 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.178360 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.178455 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:50.178464 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:50.178778 (+ 314us) server_negotiation.cc:299] Negotiation successful
0620 11:37:50.178821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 11:37:50.179020 6459 raft_consensus.cc:2454] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48cc49b9891341dbb966fe70cd7e1a09 in term 0.
I20260620 11:37:50.179251 6260 leader_election.cc:302] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4978558e594f60b960935be208a531, 48cc49b9891341dbb966fe70cd7e1a09; no voters:
I20260620 11:37:50.179289 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2ff92e4a84fb6b130e2b41e74c9a5" candidate_uuid: "48cc49b9891341dbb966fe70cd7e1a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8d1c561e2048e882688aca513314df" is_pre_election: true
I20260620 11:37:50.179374 6664 raft_consensus.cc:2781] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:50.179435 6664 raft_consensus.cc:482] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:50.179457 6593 raft_consensus.cc:2454] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48cc49b9891341dbb966fe70cd7e1a09 in term 0.
I20260620 11:37:50.179469 6664 raft_consensus.cc:3037] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:50.180195 6664 raft_consensus.cc:504] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.180348 6664 leader_election.cc:288] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [CANDIDATE]: Term 1 election: Requested vote from peers 1d4978558e594f60b960935be208a531 (127.5.122.130:37617), fc8d1c561e2048e882688aca513314df (127.5.122.131:44277)
I20260620 11:37:50.180636 6459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2ff92e4a84fb6b130e2b41e74c9a5" candidate_uuid: "48cc49b9891341dbb966fe70cd7e1a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d4978558e594f60b960935be208a531"
I20260620 11:37:50.180712 6459 raft_consensus.cc:3037] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:50.180696 6593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98a2ff92e4a84fb6b130e2b41e74c9a5" candidate_uuid: "48cc49b9891341dbb966fe70cd7e1a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc8d1c561e2048e882688aca513314df"
I20260620 11:37:50.180799 6593 raft_consensus.cc:3037] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:50.181444 6459 raft_consensus.cc:2454] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 1d4978558e594f60b960935be208a531 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48cc49b9891341dbb966fe70cd7e1a09 in term 1.
I20260620 11:37:50.181555 6593 raft_consensus.cc:2454] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48cc49b9891341dbb966fe70cd7e1a09 in term 1.
I20260620 11:37:50.181689 6260 leader_election.cc:302] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d4978558e594f60b960935be208a531, 48cc49b9891341dbb966fe70cd7e1a09; no voters:
I20260620 11:37:50.181784 6664 raft_consensus.cc:2781] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:50.182017 6664 raft_consensus.cc:686] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [term 1 LEADER]: Becoming Leader. State: Replica: 48cc49b9891341dbb966fe70cd7e1a09, State: Running, Role: LEADER
I20260620 11:37:50.182118 6664 consensus_queue.cc:237] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } }
I20260620 11:37:50.182992 6180 catalog_manager.cc:5495] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P 48cc49b9891341dbb966fe70cd7e1a09 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48cc49b9891341dbb966fe70cd7e1a09 (127.5.122.129). New cstate: current_term: 1 leader_uuid: "48cc49b9891341dbb966fe70cd7e1a09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d4978558e594f60b960935be208a531" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc8d1c561e2048e882688aca513314df" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 44277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48cc49b9891341dbb966fe70cd7e1a09" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43631 } health_report { overall_health: HEALTHY } } }
W20260620 11:37:50.204841 6640 tablet.cc:2369] T 98a2ff92e4a84fb6b130e2b41e74c9a5 P fc8d1c561e2048e882688aca513314df: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:37:50.223376 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:50.215400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.5.122.129:43631)
0620 11:37:50.215485 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:50.215490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:50.215583 (+ 93us) server_negotiation.cc:407] Connection header received
0620 11:37:50.215643 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:50.215643 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:50.215655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:50.215696 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:50.216293 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.216788 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.217582 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.217759 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.217808 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:50.220549 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:50.220564 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:50.220565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:50.220573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:50.222239 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:50.222605 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:50.222609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:50.222610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:50.222666 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:50.222871 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:50.222872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:50.222872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:50.223026 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:50.223137 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:50.223209 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 11:37:50.223272 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955470, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955470, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:37:50.247627 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:50.240530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.5.122.129:43631)
0620 11:37:50.240614 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:50.240616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:50.240632 (+ 16us) server_negotiation.cc:407] Connection header received
0620 11:37:50.240713 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:50.240714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:50.240723 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:50.240777 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:50.241347 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.241828 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.242615 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.242808 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.242866 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:50.245187 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:50.245189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:50.245189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:50.245197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:50.246569 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:50.246887 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:50.246888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:50.246888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:50.246942 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:50.247105 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:50.247105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:50.247106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:50.247199 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:50.247290 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:50.247424 (+ 134us) server_negotiation.cc:299] Negotiation successful
0620 11:37:50.247493 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6135](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:37:50.270934 6654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:50.264016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38150 (local address 127.5.122.129:43631)
0620 11:37:50.264067 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:50.264069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:50.264154 (+ 85us) server_negotiation.cc:407] Connection header received
0620 11:37:50.264202 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:50.264203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:50.264212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:50.264262 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:50.264836 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.265331 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.266135 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:50.266332 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:50.266390 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:50.268668 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:50.268670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:50.268671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:50.268679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:50.269903 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:50.270271 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:50.270272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:50.270272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:50.270326 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:50.270512 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:50.270513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:50.270513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:50.270606 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:50.270682 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:50.270764 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 11:37:50.270828 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 11:37:50.271507 6285 tablet_service.cc:2908] Rejecting scan request for tablet 98a2ff92e4a84fb6b130e2b41e74c9a5: Uninitialized: clean time has not yet been initialized
I20260620 11:37:50.272460 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 6242
I20260620 11:37:50.278308 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 6376
I20260620 11:37:50.282789 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 6510
I20260620 11:37:50.287168 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 6148
2026-06-20T11:37:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2153 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6681](info): set up 2 sockets
Jun 20 11:37:50 dist-test-slave-szdr krb5kdc[6681](info): commencing operation
krb5kdc: starting...
W20260620 11:37:51.317453 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:37:51 dist-test-slave-szdr krb5kdc[6681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:37:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:37:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:37:51.440682 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41393
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:51.523524 6694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:51.523765 6694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:51.523792 6694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:51.525074 6694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:37:51.525130 6694 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 11:37:51.525143 6694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:51.525157 6694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:51.525220 6694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:37:51.525239 6694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:37:51.526986 6694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41393
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:51.527323 6694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:51.527591 6694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:51.532701 6702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:51.532891 6694 server_base.cc:1034] running on GCE node
W20260620 11:37:51.532696 6699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:51.532696 6700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:51.533242 6694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:51.533577 6694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:51.534749 6694 hybrid_clock.cc:648] HybridClock initialized: now 1781955471534733 us; error 47 us; skew 500 ppm
Jun 20 11:37:51 dist-test-slave-szdr krb5kdc[6681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:51.541381 6694 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:37:51.542115 6694 webserver.cc:460] Webserver started at http://127.5.122.190:39265/ using document root <none> and password file <none>
I20260620 11:37:51.542357 6694 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:51.542404 6694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:51.542508 6694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:51.543622 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "d696e08d79d844c9bb59485b562fc5ec"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "aa4d69fcb3dec8d8b288d273f3c5485d"
server_key_iv: "ba471208af4220696457950d2a2fa763"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.543951 6694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "d696e08d79d844c9bb59485b562fc5ec"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "aa4d69fcb3dec8d8b288d273f3c5485d"
server_key_iv: "ba471208af4220696457950d2a2fa763"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.545369 6694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:37:51.546241 6709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:51.546435 6694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:37:51.546545 6694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "d696e08d79d844c9bb59485b562fc5ec"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "aa4d69fcb3dec8d8b288d273f3c5485d"
server_key_iv: "ba471208af4220696457950d2a2fa763"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.546610 6694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:51.566826 6694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:51.568837 6694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:51.568960 6694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:51.573479 6694 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:41393
I20260620 11:37:51.573575 6771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:41393 every 8 connection(s)
I20260620 11:37:51.573927 6694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:37:51.574684 6772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:51.575836 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 6694
I20260620 11:37:51.575917 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:37:51.576094 5610 external_mini_cluster.cc:1428] Setting key 806743d699f4e2f298a2f859d9ef6277
I20260620 11:37:51.577383 6772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec: Bootstrap starting.
I20260620 11:37:51.578225 6772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:51.578555 6772 log.cc:824] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:51.579247 6772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec: No bootstrap required, opened a new log
I20260620 11:37:51.581188 6772 raft_consensus.cc:348] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d696e08d79d844c9bb59485b562fc5ec" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41393 } }
I20260620 11:37:51.581377 6772 raft_consensus.cc:374] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:51.581406 6772 raft_consensus.cc:729] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d696e08d79d844c9bb59485b562fc5ec, State: Initialized, Role: FOLLOWER
I20260620 11:37:51.581561 6772 consensus_queue.cc:260] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d696e08d79d844c9bb59485b562fc5ec" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41393 } }
I20260620 11:37:51.581636 6772 raft_consensus.cc:388] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:37:51.581694 6772 raft_consensus.cc:482] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:37:51.581743 6772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:51.582424 6772 raft_consensus.cc:504] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d696e08d79d844c9bb59485b562fc5ec" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41393 } }
I20260620 11:37:51.582551 6772 leader_election.cc:302] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d696e08d79d844c9bb59485b562fc5ec; no voters:
I20260620 11:37:51.582729 6772 leader_election.cc:288] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:37:51.582804 6777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 1 FOLLOWER]: Leader election won for term 1
Jun 20 11:37:51 dist-test-slave-szdr krb5kdc[6681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:51.582986 6777 raft_consensus.cc:686] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [term 1 LEADER]: Becoming Leader. State: Replica: d696e08d79d844c9bb59485b562fc5ec, State: Running, Role: LEADER
I20260620 11:37:51.583005 6772 sys_catalog.cc:564] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:37:51.583606 6777 consensus_queue.cc:237] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d696e08d79d844c9bb59485b562fc5ec" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41393 } }
I20260620 11:37:51.584034 6779 sys_catalog.cc:455] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader d696e08d79d844c9bb59485b562fc5ec. Latest consensus state: current_term: 1 leader_uuid: "d696e08d79d844c9bb59485b562fc5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d696e08d79d844c9bb59485b562fc5ec" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41393 } } }
I20260620 11:37:51.584127 6779 sys_catalog.cc:458] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:51.584293 6778 sys_catalog.cc:455] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d696e08d79d844c9bb59485b562fc5ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d696e08d79d844c9bb59485b562fc5ec" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41393 } } }
I20260620 11:37:51.584561 6778 sys_catalog.cc:458] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:51.584432 6792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:37:51.585094 6792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:37:51.586817 6792 catalog_manager.cc:1347] Generated new cluster ID: 402eba4c86c04a35ba92e3f6a465e567
I20260620 11:37:51.586884 6792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:37:51.587731 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.576640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.5.122.190:41393)
0620 11:37:51.576812 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:51.576820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:51.576847 (+ 27us) server_negotiation.cc:407] Connection header received
0620 11:37:51.577258 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:51.577264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:51.577322 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:51.577442 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:51.578253 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.579272 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.580117 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.580566 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.580691 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.583601 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:51.583606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:51.583606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:51.583619 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:51.585933 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.586444 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.586447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.586448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.586548 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.586766 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.586767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.586767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.587080 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:51.587227 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:51.587365 (+ 138us) server_negotiation.cc:299] Negotiation successful
0620 11:37:51.587509 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260620 11:37:51.602424 6792 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:37:51.603070 6792 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:37:51.608590 6792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec: Generated new TSK 0
I20260620 11:37:51.608827 6792 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:37:51.636363 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41393
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:51.720903 6798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:51.721180 6798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:51.721235 6798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:51.722599 6798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:51.722643 6798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:51.722664 6798 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:37:51.722683 6798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:37:51.724218 6798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:41393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:51.724466 6798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:51.724702 6798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:51.729936 6804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:51.730005 6803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:51.729936 6806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:51.730217 6798 server_base.cc:1034] running on GCE node
I20260620 11:37:51.730460 6798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:51.730713 6798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:51.731882 6798 hybrid_clock.cc:648] HybridClock initialized: now 1781955471731870 us; error 30 us; skew 500 ppm
Jun 20 11:37:51 dist-test-slave-szdr krb5kdc[6681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:51.734583 6798 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:37:51.735038 6798 webserver.cc:460] Webserver started at http://127.5.122.129:46343/ using document root <none> and password file <none>
I20260620 11:37:51.735275 6798 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:51.735350 6798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:51.735503 6798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:51.736622 6798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "e3b81650d705441d9b366ae7607b55a7"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "1c28bd1150aa24a7ac8649134e9748b9"
server_key_iv: "24731bab1ae230631df1826dd7383949"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.737018 6798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "e3b81650d705441d9b366ae7607b55a7"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "1c28bd1150aa24a7ac8649134e9748b9"
server_key_iv: "24731bab1ae230631df1826dd7383949"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.738447 6798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:37:51.739171 6813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:51.739360 6798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 11:37:51.739462 6798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "e3b81650d705441d9b366ae7607b55a7"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "1c28bd1150aa24a7ac8649134e9748b9"
server_key_iv: "24731bab1ae230631df1826dd7383949"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.739542 6798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:51.765728 6798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:51.768388 6798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:51.768573 6798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:51.769161 6798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:51.769234 6798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:51.769285 6798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:51.769325 6798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:51.776279 6798 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:46239
I20260620 11:37:51.776389 6928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:46239 every 8 connection(s)
I20260620 11:37:51.776715 6798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
Jun 20 11:37:51 dist-test-slave-szdr krb5kdc[6681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:51.780148 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.769765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:34761 (local address 127.5.122.190:41393)
0620 11:37:51.769823 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:51.769825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:51.770144 (+ 319us) server_negotiation.cc:407] Connection header received
0620 11:37:51.770738 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:51.770740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:51.770753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:51.770802 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:51.771709 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.772612 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.774237 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.774537 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.774630 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.777399 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:51.777401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:51.777401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:51.777411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:51.778677 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.779128 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.779129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.779130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.779195 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.779463 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.779464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.779465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.779590 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:51.779665 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:51.779960 (+ 295us) server_negotiation.cc:299] Negotiation successful
0620 11:37:51.780054 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:37:51.780337 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.769755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41393 (local address 127.5.122.129:34761)
0620 11:37:51.769951 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:51.770043 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:51.770617 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:51.770885 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:51.770890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:51.771303 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:51.771614 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.771620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.772701 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.772703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.774154 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.774157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.774640 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.774642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.774807 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.775460 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:51.775481 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:51.777282 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:51.778805 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.778813 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.778815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.779044 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.779266 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.779267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.779267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.779388 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.779657 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:51.779663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:51.779821 (+ 158us) client_negotiation.cc:769] Sending connection context
0620 11:37:51.779928 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 11:37:51.780095 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260620 11:37:51.780827 6929 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41393
I20260620 11:37:51.780920 6929 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:51.781114 6929 heartbeater.cc:511] Master 127.5.122.190:41393 requested a full tablet report, sending...
I20260620 11:37:51.781759 6725 ts_manager.cc:194] Registered new tserver with Master: e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239)
I20260620 11:37:51.782126 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 6798
I20260620 11:37:51.782225 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:37:51.782390 5610 external_mini_cluster.cc:1428] Setting key 3602973b7a800e8d86ac633964bd6293
I20260620 11:37:51.782749 6725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:34761
I20260620 11:37:51.788061 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.781924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41393 (local address 127.5.122.129:53459)
0620 11:37:51.782079 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:51.782089 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:51.782173 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:51.782350 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:51.782351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:51.782585 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:51.782651 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.782654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.783495 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.783496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.784088 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.784090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.784438 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.784439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.784514 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.784982 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:51.784987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:51.785284 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:51.787114 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.787114 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.787115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.787302 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.787506 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.787506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.787507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.787556 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.787833 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:51.787833 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:51.787868 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 11:37:51.787915 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:37:51.787991 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260620 11:37:51.788179 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.781983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:53459 (local address 127.5.122.190:41393)
0620 11:37:51.782026 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:51.782028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:51.782157 (+ 129us) server_negotiation.cc:407] Connection header received
0620 11:37:51.782245 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:51.782246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:51.782255 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:51.782285 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:51.782750 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.783388 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.784167 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.784363 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.784440 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.785389 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:51.785391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:51.785391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:51.785399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:51.787032 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.787375 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.787376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.787376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.787440 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.787613 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.787615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.787615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.787762 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:51.787847 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:51.787949 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 11:37:51.788034 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:37:51.809049 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41393
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:51.894789 6938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:51.895030 6938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:51.895058 6938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:51.896440 6938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:51.896493 6938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:51.896517 6938 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:37:51.896536 6938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:37:51.898269 6938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:41393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:51.898590 6938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:51.898851 6938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:51.904050 6946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:51.904049 6944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:51.904250 6943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:51.904345 6938 server_base.cc:1034] running on GCE node
I20260620 11:37:51.904621 6938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:51.904914 6938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:51.906037 6938 hybrid_clock.cc:648] HybridClock initialized: now 1781955471906011 us; error 40 us; skew 500 ppm
Jun 20 11:37:51 dist-test-slave-szdr krb5kdc[6681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:51.908815 6938 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:37:51.909314 6938 webserver.cc:460] Webserver started at http://127.5.122.130:37391/ using document root <none> and password file <none>
I20260620 11:37:51.909512 6938 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:51.909554 6938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:51.909648 6938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:51.910816 6938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "f379b6ea23a04ffe9a852e214e6f7998"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "0f1750d9c6f4ab8b8528c6e4ca9ff5ec"
server_key_iv: "9d7a2fc19e0be104a7d989b6de3f57c8"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.911134 6938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "f379b6ea23a04ffe9a852e214e6f7998"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "0f1750d9c6f4ab8b8528c6e4ca9ff5ec"
server_key_iv: "9d7a2fc19e0be104a7d989b6de3f57c8"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.912436 6938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 11:37:51.913187 6953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:51.913403 6938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:37:51.913496 6938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "f379b6ea23a04ffe9a852e214e6f7998"
format_stamp: "Formatted at 2026-06-20 11:37:51 on dist-test-slave-szdr"
server_key: "0f1750d9c6f4ab8b8528c6e4ca9ff5ec"
server_key_iv: "9d7a2fc19e0be104a7d989b6de3f57c8"
server_key_version: "encryptionkey@0"
I20260620 11:37:51.913552 6938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:51.923681 6938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:51.926327 6938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:51.926447 6938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:51.926962 6938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:51.927002 6938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:51.927027 6938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:51.927065 6938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:51.934208 6938 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:37473
I20260620 11:37:51.934257 7068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:37473 every 8 connection(s)
I20260620 11:37:51.934595 6938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
Jun 20 11:37:51 dist-test-slave-szdr krb5kdc[6681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:51.938194 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.927465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:36373 (local address 127.5.122.190:41393)
0620 11:37:51.927543 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:51.927547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:51.927730 (+ 183us) server_negotiation.cc:407] Connection header received
0620 11:37:51.928256 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:51.928257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:51.928270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:51.928358 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:51.929271 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.930131 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.931890 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.932195 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.932253 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.935260 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:51.935262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:51.935262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:51.935271 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:51.936722 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.937177 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.937179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.937179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.937244 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.937505 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.937506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.937507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.937629 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:51.937708 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:51.938007 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 11:37:51.938095 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:37:51.938426 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.927434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41393 (local address 127.5.122.130:36373)
0620 11:37:51.927594 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:51.927636 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:51.928139 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:51.928457 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:51.928464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:51.928857 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:51.929165 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.929173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.930238 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.930239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.931775 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.931780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.932288 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.932289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.932456 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.933143 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:51.933164 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:51.935154 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:51.936854 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.936861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.936863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.937097 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.937318 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.937319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.937319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.937436 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.937699 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:51.937705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:51.937854 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 11:37:51.937966 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 11:37:51.938178 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260620 11:37:51.938879 7069 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41393
I20260620 11:37:51.938971 7069 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:51.939158 7069 heartbeater.cc:511] Master 127.5.122.190:41393 requested a full tablet report, sending...
I20260620 11:37:51.939535 6726 ts_manager.cc:194] Registered new tserver with Master: f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130:37473)
I20260620 11:37:51.940063 6726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:36373
I20260620 11:37:51.944253 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 6938
I20260620 11:37:51.944373 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:37:51.944582 5610 external_mini_cluster.cc:1428] Setting key 253d7af3ecde81a1af02eccee0b5dfc6
I20260620 11:37:51.945662 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.940057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:46665 (local address 127.5.122.190:41393)
0620 11:37:51.940096 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:51.940097 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:51.940188 (+ 91us) server_negotiation.cc:407] Connection header received
0620 11:37:51.940309 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:51.940310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:51.940320 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:51.940377 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:51.940710 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.941184 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.941887 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.942156 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.942241 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.943202 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:51.943204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:51.943204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:51.943212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:51.944397 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.944784 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.944785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.944785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.944850 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:51.945055 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:51.945056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:51.945057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:51.945256 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:51.945341 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:51.945531 (+ 190us) server_negotiation.cc:299] Negotiation successful
0620 11:37:51.945599 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:37:51.945775 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:51.939965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41393 (local address 127.5.122.130:46665)
0620 11:37:51.940115 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:51.940127 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:51.940214 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:51.940383 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:51.940384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:51.940568 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:51.940621 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.940622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.941292 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.941293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.941800 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:51.941802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:51.942243 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:51.942244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:51.942355 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:51.942836 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:51.942840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:51.943110 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:51.944478 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.944479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.944480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.944703 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.944919 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:51.944920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:51.944921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:51.944988 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:51.945369 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:51.945370 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:51.945422 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 11:37:51.945502 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 11:37:51.945618 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:37:51.973781 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41393
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:52.060927 7078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:52.061235 7078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:52.061280 7078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:52.062631 7078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:52.062714 7078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:52.062749 7078 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:37:52.062783 7078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:37:52.064477 7078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:41393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:52.064810 7078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:52.065078 7078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:52.070616 7086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:52.070642 7083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:52.070627 7084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:52.070838 7078 server_base.cc:1034] running on GCE node
I20260620 11:37:52.071066 7078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:52.071324 7078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:52.072477 7078 hybrid_clock.cc:648] HybridClock initialized: now 1781955472072462 us; error 30 us; skew 500 ppm
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[6681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955472, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:52.075196 7078 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:37:52.075703 7078 webserver.cc:460] Webserver started at http://127.5.122.131:43659/ using document root <none> and password file <none>
I20260620 11:37:52.075973 7078 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:52.076021 7078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:52.076184 7078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:52.077318 7078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "149202ea7b1341ccb41b07b9e36d1752"
format_stamp: "Formatted at 2026-06-20 11:37:52 on dist-test-slave-szdr"
server_key: "3abe6254f77957b2de7060095358a918"
server_key_iv: "75206354fd35b8c9ee2383e35d8af7a8"
server_key_version: "encryptionkey@0"
I20260620 11:37:52.077705 7078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "149202ea7b1341ccb41b07b9e36d1752"
format_stamp: "Formatted at 2026-06-20 11:37:52 on dist-test-slave-szdr"
server_key: "3abe6254f77957b2de7060095358a918"
server_key_iv: "75206354fd35b8c9ee2383e35d8af7a8"
server_key_version: "encryptionkey@0"
I20260620 11:37:52.079224 7078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:37:52.080004 7093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:52.080199 7078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:37:52.080286 7078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "149202ea7b1341ccb41b07b9e36d1752"
format_stamp: "Formatted at 2026-06-20 11:37:52 on dist-test-slave-szdr"
server_key: "3abe6254f77957b2de7060095358a918"
server_key_iv: "75206354fd35b8c9ee2383e35d8af7a8"
server_key_version: "encryptionkey@0"
I20260620 11:37:52.080380 7078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:52.099079 7078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:52.101162 7078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:52.101277 7078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:52.101941 7078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:52.102020 7078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:52.102056 7078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:52.102074 7078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[6681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955472, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:52.109501 7078 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:45363
I20260620 11:37:52.109566 7208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:45363 every 8 connection(s)
I20260620 11:37:52.109892 7078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:37:52.110347 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 7078
I20260620 11:37:52.110455 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:37:52.110613 5610 external_mini_cluster.cc:1428] Setting key 1094487edd537d98f45a4a2379728332
I20260620 11:37:52.112136 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.102351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:52325 (local address 127.5.122.190:41393)
0620 11:37:52.102430 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.102433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.102669 (+ 236us) server_negotiation.cc:407] Connection header received
0620 11:37:52.103154 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.103156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.103168 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.103308 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:52.103937 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.104447 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.105465 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.105680 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.105759 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.109115 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:52.109117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:52.109117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:52.109126 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:52.110903 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.111368 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.111369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.111369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.111433 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.111665 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.111666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.111666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.111787 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:52.111837 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.112017 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.112075 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 11:37:52.112371 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.102366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41393 (local address 127.5.122.131:52325)
0620 11:37:52.102538 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.102579 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.103063 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.103310 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.103314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.103618 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:52.103842 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.103847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.104551 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.104552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.105359 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.105361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.105769 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.105770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.105896 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.106489 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:52.106502 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:52.109022 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:52.110967 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:52.110971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:52.110973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:52.111269 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:52.111507 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:52.111508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:52.111508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:52.111606 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:52.111827 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:52.111830 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:52.111934 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.112008 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.112121 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260620 11:37:52.112821 7209 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41393
I20260620 11:37:52.112901 7209 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:52.113088 7209 heartbeater.cc:511] Master 127.5.122.190:41393 requested a full tablet report, sending...
I20260620 11:37:52.113451 6726 ts_manager.cc:194] Registered new tserver with Master: 149202ea7b1341ccb41b07b9e36d1752 (127.5.122.131:45363)
I20260620 11:37:52.113934 6726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:52325
I20260620 11:37:52.118676 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.113804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:37263 (local address 127.5.122.190:41393)
0620 11:37:52.113854 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.113857 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.113927 (+ 70us) server_negotiation.cc:407] Connection header received
0620 11:37:52.114018 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.114019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.114029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.114075 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:52.114507 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.114952 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.115633 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.115805 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.115854 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.116690 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:52.116691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:52.116691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:52.116697 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:52.117806 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.118151 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.118151 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.118151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.118194 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.118365 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.118365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.118365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.118445 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:52.118488 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.118571 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.118622 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 11:37:52.118688 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.113747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41393 (local address 127.5.122.131:37263)
0620 11:37:52.113875 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.113885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.113940 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.114144 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.114145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.114367 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:52.114431 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.114433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.115042 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.115043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.115563 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.115564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.115869 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.115869 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.115923 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.116270 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:52.116274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:52.116620 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:52.117881 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:52.117881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:52.117882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:52.118097 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:52.118269 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:52.118270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:52.118270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:52.118320 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:52.118478 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:52.118479 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:52.118509 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.118554 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.118624 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":126,"threads_started":1}
I20260620 11:37:52.122202 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[6681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[6681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:52.145349 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.138000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.5.122.190:41393)
0620 11:37:52.138067 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.138069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.138080 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:37:52.138141 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.138142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.138151 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.138204 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:52.138755 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.139375 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.140243 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.140454 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.140509 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.143075 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:52.143077 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:52.143078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:52.143085 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:52.144364 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.144706 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.144707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.144707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.144761 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.144927 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.144927 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.144927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.145030 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:52.145101 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.145196 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.145283 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:37:52.147292 6726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:37:52.148932 6726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:52.155916 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.151623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.5.122.131:45363)
0620 11:37:52.151802 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.151807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.151940 (+ 133us) server_negotiation.cc:407] Connection header received
0620 11:37:52.152074 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.152077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.152096 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.152167 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:52.152506 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.153619 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.154847 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.155418 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.155524 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.155594 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.155763 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.155813 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260620 11:37:52.155900 7227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.151745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:37473 (local address 127.0.0.1:54946)
0620 11:37:52.151943 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.151953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.151999 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.152308 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.152309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.152311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:52.152401 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.152403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.153333 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.153334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.154620 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.154623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.155605 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.155606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.155687 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.155693 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.155725 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.155803 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":9,"thread_start_us":98,"threads_started":1}
I20260620 11:37:52.156018 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.151538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46239 (local address 127.0.0.1:57752)
0620 11:37:52.151882 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.151894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.151960 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.152318 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.152319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.152321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:52.152411 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.152413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.153692 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.153693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.154830 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.154833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.155720 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.155721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.155780 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.155783 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.155837 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.155884 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":32,"thread_start_us":79,"threads_started":1}
I20260620 11:37:52.156139 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.151727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.5.122.129:46239)
0620 11:37:52.152108 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.152112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.152122 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:52.152162 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.152164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.152179 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.152251 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:52.152493 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.153598 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.154901 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.155640 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.155705 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.155766 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.155851 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.155886 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":121,"threads_started":1}
I20260620 11:37:52.156318 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.151980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54946 (local address 127.5.122.130:37473)
0620 11:37:52.152148 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.152152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.152162 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:52.152199 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.152201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.152215 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.152477 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:52.152532 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.153230 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.154716 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.155501 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.156058 (+ 557us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.156139 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.156166 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.156213 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260620 11:37:52.156665 7226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.151538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:45363 (local address 127.0.0.1:58684)
0620 11:37:52.151819 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.151866 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.151977 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.152301 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.152304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.152306 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:52.152401 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.152404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.153718 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.153719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.154754 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.154756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.155528 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.155529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.155618 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.155633 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.156545 (+ 912us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.156592 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":9,"thread_start_us":65,"threads_started":1}
I20260620 11:37:52.157014 6863 tablet_service.cc:1467] Processing CreateTablet for tablet ddca90b4abf841e29b1f1195956d1586 (DEFAULT_TABLE table=test-table [id=af05af6e73ba4b03bf45868f59d960e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:52.157428 6863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddca90b4abf841e29b1f1195956d1586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.157326 7143 tablet_service.cc:1467] Processing CreateTablet for tablet ddca90b4abf841e29b1f1195956d1586 (DEFAULT_TABLE table=test-table [id=af05af6e73ba4b03bf45868f59d960e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:52.157634 7143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddca90b4abf841e29b1f1195956d1586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.158396 7003 tablet_service.cc:1467] Processing CreateTablet for tablet ddca90b4abf841e29b1f1195956d1586 (DEFAULT_TABLE table=test-table [id=af05af6e73ba4b03bf45868f59d960e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:52.158735 7003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddca90b4abf841e29b1f1195956d1586. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.161197 7232 tablet_bootstrap.cc:492] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Bootstrap starting.
I20260620 11:37:52.161473 7233 tablet_bootstrap.cc:492] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Bootstrap starting.
I20260620 11:37:52.162115 7234 tablet_bootstrap.cc:492] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Bootstrap starting.
I20260620 11:37:52.162220 7233 tablet_bootstrap.cc:654] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.162235 7232 tablet_bootstrap.cc:654] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.162533 7233 log.cc:824] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:52.162595 7232 log.cc:824] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:52.162788 7234 tablet_bootstrap.cc:654] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.163133 7234 log.cc:824] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:52.163367 7233 tablet_bootstrap.cc:492] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: No bootstrap required, opened a new log
I20260620 11:37:52.163452 7233 ts_tablet_manager.cc:1397] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:52.163823 7234 tablet_bootstrap.cc:492] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: No bootstrap required, opened a new log
I20260620 11:37:52.163878 7234 ts_tablet_manager.cc:1397] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:52.164924 7233 raft_consensus.cc:348] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.165102 7233 raft_consensus.cc:374] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.165149 7233 raft_consensus.cc:729] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b81650d705441d9b366ae7607b55a7, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.165284 7233 consensus_queue.cc:260] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.165571 7233 ts_tablet_manager.cc:1428] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:37:52.165686 6929 heartbeater.cc:503] Master 127.5.122.190:41393 was elected leader, sending a full tablet report...
I20260620 11:37:52.165472 7234 raft_consensus.cc:348] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.165856 7234 raft_consensus.cc:374] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.165900 7234 raft_consensus.cc:729] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379b6ea23a04ffe9a852e214e6f7998, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.166096 7232 tablet_bootstrap.cc:492] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: No bootstrap required, opened a new log
I20260620 11:37:52.166082 7234 consensus_queue.cc:260] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.166203 7232 ts_tablet_manager.cc:1397] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260620 11:37:52.166380 7234 ts_tablet_manager.cc:1428] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 11:37:52.166424 7069 heartbeater.cc:503] Master 127.5.122.190:41393 was elected leader, sending a full tablet report...
I20260620 11:37:52.168076 7232 raft_consensus.cc:348] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.168269 7232 raft_consensus.cc:374] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.168319 7232 raft_consensus.cc:729] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149202ea7b1341ccb41b07b9e36d1752, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.168440 7232 consensus_queue.cc:260] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.168736 7232 ts_tablet_manager.cc:1428] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 11:37:52.168818 7209 heartbeater.cc:503] Master 127.5.122.190:41393 was elected leader, sending a full tablet report...
W20260620 11:37:52.185173 7070 tablet.cc:2369] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:37:52.225852 7240 raft_consensus.cc:482] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:52.226012 7240 raft_consensus.cc:504] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.226310 7240 leader_election.cc:288] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239), f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130:37473)
I20260620 11:37:52.229632 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.226684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:45893 (local address 127.5.122.129:46239)
0620 11:37:52.226728 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.226730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.226740 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:52.226807 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.226808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.226818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.226883 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:52.227076 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.227838 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.228864 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.229246 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.229305 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.229312 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.229510 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.229558 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:37:52.229651 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.226775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:44363 (local address 127.5.122.130:37473)
0620 11:37:52.226823 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.226825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.226953 (+ 128us) server_negotiation.cc:407] Connection header received
0620 11:37:52.226984 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.226985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.226995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.227054 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:52.227231 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.227929 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.228864 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.229257 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.229351 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.229360 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.229549 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.229592 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:37:52.229633 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.226559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46239 (local address 127.5.122.131:45893)
0620 11:37:52.226643 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.226658 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.226726 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.226911 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.226913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.226915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:52.226993 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.226995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.227918 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.227919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.228758 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.228760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.229335 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.229336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.229433 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.229437 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.229480 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.229540 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 11:37:52.229648 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.226674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:37473 (local address 127.5.122.131:44363)
0620 11:37:52.226850 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.226860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.226908 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.227076 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.227077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.227079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:52.227148 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.227150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.228021 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.228021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.228757 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.228759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.229371 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.229372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.229448 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.229451 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.229525 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.229564 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":129,"threads_started":1}
I20260620 11:37:52.230105 6864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddca90b4abf841e29b1f1195956d1586" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b81650d705441d9b366ae7607b55a7" is_pre_election: true
I20260620 11:37:52.230134 7023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddca90b4abf841e29b1f1195956d1586" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f379b6ea23a04ffe9a852e214e6f7998" is_pre_election: true
I20260620 11:37:52.230295 7023 raft_consensus.cc:2454] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 0.
I20260620 11:37:52.230265 6864 raft_consensus.cc:2454] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 0.
I20260620 11:37:52.230543 7096 leader_election.cc:302] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149202ea7b1341ccb41b07b9e36d1752, e3b81650d705441d9b366ae7607b55a7; no voters:
I20260620 11:37:52.230726 7240 raft_consensus.cc:2781] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:52.230806 7240 raft_consensus.cc:482] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:52.230825 7240 raft_consensus.cc:3037] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.231576 7240 raft_consensus.cc:504] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.231721 7240 leader_election.cc:288] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 election: Requested vote from peers e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239), f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130:37473)
I20260620 11:37:52.231936 6864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddca90b4abf841e29b1f1195956d1586" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b81650d705441d9b366ae7607b55a7"
I20260620 11:37:52.232024 6864 raft_consensus.cc:3037] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.232000 7023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddca90b4abf841e29b1f1195956d1586" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f379b6ea23a04ffe9a852e214e6f7998"
I20260620 11:37:52.232110 7023 raft_consensus.cc:3037] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.232832 6864 raft_consensus.cc:2454] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 1.
I20260620 11:37:52.232867 7023 raft_consensus.cc:2454] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 1.
I20260620 11:37:52.233036 7096 leader_election.cc:302] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149202ea7b1341ccb41b07b9e36d1752, f379b6ea23a04ffe9a852e214e6f7998; no voters:
I20260620 11:37:52.233150 7240 raft_consensus.cc:2781] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:52.233311 7240 raft_consensus.cc:686] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 1 LEADER]: Becoming Leader. State: Replica: 149202ea7b1341ccb41b07b9e36d1752, State: Running, Role: LEADER
I20260620 11:37:52.233439 7240 consensus_queue.cc:237] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.234367 6725 catalog_manager.cc:5495] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 reported cstate change: term changed from 0 to 1, leader changed from <none> to 149202ea7b1341ccb41b07b9e36d1752 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "149202ea7b1341ccb41b07b9e36d1752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } health_report { overall_health: HEALTHY } } }
I20260620 11:37:52.252261 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.249535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.5.122.131:45363)
0620 11:37:52.249594 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.249596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.249604 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:37:52.249726 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.249727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.249744 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.249806 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:52.250174 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.250667 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.251445 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.251633 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.251695 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.251880 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:52.251978 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:52.252044 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.252090 (+ 46us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.252110 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 11:37:52.254547 7023 raft_consensus.cc:1261] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Refusing update from remote peer 149202ea7b1341ccb41b07b9e36d1752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:52.254597 6864 raft_consensus.cc:1261] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Refusing update from remote peer 149202ea7b1341ccb41b07b9e36d1752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:52.254968 7240 consensus_queue.cc:1035] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:52.255218 7242 consensus_queue.cc:1035] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:52.258708 7247 mvcc.cc:205] Tried to move back new op lower bound from 7298889614351323136 to 7298889614268903424. Current Snapshot: MvccSnapshot[applied={T|T < 7298889614351323136}]
I20260620 11:37:52.261479 6725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:52.261586 6725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:52.262494 6725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec: Sending DeleteTablet for 3 replicas of tablet ddca90b4abf841e29b1f1195956d1586
I20260620 11:37:52.262842 6863 tablet_service.cc:1514] Processing DeleteTablet for tablet ddca90b4abf841e29b1f1195956d1586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:52 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:57752
I20260620 11:37:52.262884 7003 tablet_service.cc:1514] Processing DeleteTablet for tablet ddca90b4abf841e29b1f1195956d1586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:52 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:54946
I20260620 11:37:52.262919 7143 tablet_service.cc:1514] Processing DeleteTablet for tablet ddca90b4abf841e29b1f1195956d1586 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:52 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58684
I20260620 11:37:52.263322 7257 tablet_replica.cc:330] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: stopping tablet replica
I20260620 11:37:52.263420 7257 raft_consensus.cc:2229] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:37:52.263437 6725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 11:37:52.263530 7257 raft_consensus.cc:2258] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 11:37:52.263585 6725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:52.263705 7255 tablet_replica.cc:330] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: stopping tablet replica
I20260620 11:37:52.263825 7255 raft_consensus.cc:2229] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:52.263942 7255 raft_consensus.cc:2258] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:52.264117 7257 ts_tablet_manager.cc:1905] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:52.265301 7255 ts_tablet_manager.cc:1905] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:52.265378 7143 tablet_service.cc:1467] Processing CreateTablet for tablet c5192aade1ba4c7fbcff1ca46e8ea84a (DEFAULT_TABLE table=test-table [id=a02cdde9173448adb3c026ce6c546fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:52.265498 7143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5192aade1ba4c7fbcff1ca46e8ea84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.265983 7257 ts_tablet_manager.cc:1918] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:37:52.266058 7257 log.cc:1196] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/ddca90b4abf841e29b1f1195956d1586
I20260620 11:37:52.266322 7257 ts_tablet_manager.cc:1939] T ddca90b4abf841e29b1f1195956d1586 P 149202ea7b1341ccb41b07b9e36d1752: Deleting consensus metadata
I20260620 11:37:52.266335 6863 tablet_service.cc:1467] Processing CreateTablet for tablet c5192aade1ba4c7fbcff1ca46e8ea84a (DEFAULT_TABLE table=test-table [id=a02cdde9173448adb3c026ce6c546fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:52.266754 6711 catalog_manager.cc:4841] TS 149202ea7b1341ccb41b07b9e36d1752 (127.5.122.131:45363): tablet ddca90b4abf841e29b1f1195956d1586 (table test-table [id=af05af6e73ba4b03bf45868f59d960e7]) successfully deleted
I20260620 11:37:52.266794 7256 tablet_replica.cc:330] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: stopping tablet replica
I20260620 11:37:52.266897 7256 raft_consensus.cc:2229] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:52.266942 7003 tablet_service.cc:1467] Processing CreateTablet for tablet c5192aade1ba4c7fbcff1ca46e8ea84a (DEFAULT_TABLE table=test-table [id=a02cdde9173448adb3c026ce6c546fe4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:52.267000 7256 raft_consensus.cc:2258] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:52.267063 7003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5192aade1ba4c7fbcff1ca46e8ea84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.267391 7256 ts_tablet_manager.cc:1905] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:52.268698 7234 tablet_bootstrap.cc:492] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: Bootstrap starting.
I20260620 11:37:52.268699 7256 ts_tablet_manager.cc:1918] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:37:52.268850 7256 log.cc:1196] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/ddca90b4abf841e29b1f1195956d1586
I20260620 11:37:52.269037 7256 ts_tablet_manager.cc:1939] T ddca90b4abf841e29b1f1195956d1586 P f379b6ea23a04ffe9a852e214e6f7998: Deleting consensus metadata
I20260620 11:37:52.269201 7234 tablet_bootstrap.cc:654] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.269677 6712 catalog_manager.cc:4841] TS f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130:37473): tablet ddca90b4abf841e29b1f1195956d1586 (table test-table [id=af05af6e73ba4b03bf45868f59d960e7]) successfully deleted
I20260620 11:37:52.270073 6863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5192aade1ba4c7fbcff1ca46e8ea84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.270836 7255 ts_tablet_manager.cc:1918] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:37:52.270907 7255 log.cc:1196] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/ddca90b4abf841e29b1f1195956d1586
I20260620 11:37:52.270955 7232 tablet_bootstrap.cc:492] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Bootstrap starting.
I20260620 11:37:52.271158 7255 ts_tablet_manager.cc:1939] T ddca90b4abf841e29b1f1195956d1586 P e3b81650d705441d9b366ae7607b55a7: Deleting consensus metadata
I20260620 11:37:52.271369 7232 tablet_bootstrap.cc:654] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.271438 7233 tablet_bootstrap.cc:492] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Bootstrap starting.
I20260620 11:37:52.271875 7233 tablet_bootstrap.cc:654] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.272111 7234 tablet_bootstrap.cc:492] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: No bootstrap required, opened a new log
I20260620 11:37:52.272164 7234 ts_tablet_manager.cc:1397] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260620 11:37:52.272325 7234 raft_consensus.cc:348] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.272397 7234 raft_consensus.cc:374] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.272419 7234 raft_consensus.cc:729] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379b6ea23a04ffe9a852e214e6f7998, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.272410 7233 tablet_bootstrap.cc:492] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: No bootstrap required, opened a new log
I20260620 11:37:52.272452 7233 ts_tablet_manager.cc:1397] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:52.272467 7234 consensus_queue.cc:260] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.272483 6712 catalog_manager.cc:4841] TS e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239): tablet ddca90b4abf841e29b1f1195956d1586 (table test-table [id=af05af6e73ba4b03bf45868f59d960e7]) successfully deleted
I20260620 11:37:52.272555 7234 ts_tablet_manager.cc:1428] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:52.272607 7233 raft_consensus.cc:348] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.272672 7233 raft_consensus.cc:374] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.272696 7233 raft_consensus.cc:729] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b81650d705441d9b366ae7607b55a7, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.272749 7233 consensus_queue.cc:260] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.272845 7233 ts_tablet_manager.cc:1428] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:52.273346 7232 tablet_bootstrap.cc:492] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: No bootstrap required, opened a new log
I20260620 11:37:52.273427 7232 ts_tablet_manager.cc:1397] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:52.273581 7232 raft_consensus.cc:348] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.273701 7232 raft_consensus.cc:374] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.273725 7232 raft_consensus.cc:729] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149202ea7b1341ccb41b07b9e36d1752, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.273767 7232 consensus_queue.cc:260] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.273851 7232 ts_tablet_manager.cc:1428] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260620 11:37:52.277516 6930 tablet.cc:2369] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:37:52.288321 7240 raft_consensus.cc:482] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:52.288435 7240 raft_consensus.cc:504] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.288587 7240 leader_election.cc:288] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239), f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130:37473)
I20260620 11:37:52.288866 7023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5192aade1ba4c7fbcff1ca46e8ea84a" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f379b6ea23a04ffe9a852e214e6f7998" is_pre_election: true
I20260620 11:37:52.288875 6864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5192aade1ba4c7fbcff1ca46e8ea84a" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b81650d705441d9b366ae7607b55a7" is_pre_election: true
I20260620 11:37:52.288983 6864 raft_consensus.cc:2454] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 0.
I20260620 11:37:52.288983 7023 raft_consensus.cc:2454] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 0.
I20260620 11:37:52.289139 7096 leader_election.cc:302] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149202ea7b1341ccb41b07b9e36d1752, e3b81650d705441d9b366ae7607b55a7; no voters:
I20260620 11:37:52.289251 7240 raft_consensus.cc:2781] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:52.289294 7240 raft_consensus.cc:482] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:52.289323 7240 raft_consensus.cc:3037] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.290076 7240 raft_consensus.cc:504] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.290230 7240 leader_election.cc:288] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 election: Requested vote from peers e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239), f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130:37473)
I20260620 11:37:52.290393 6864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5192aade1ba4c7fbcff1ca46e8ea84a" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b81650d705441d9b366ae7607b55a7"
I20260620 11:37:52.290427 7023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5192aade1ba4c7fbcff1ca46e8ea84a" candidate_uuid: "149202ea7b1341ccb41b07b9e36d1752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f379b6ea23a04ffe9a852e214e6f7998"
I20260620 11:37:52.290485 6864 raft_consensus.cc:3037] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.290493 7023 raft_consensus.cc:3037] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.291244 6864 raft_consensus.cc:2454] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 1.
I20260620 11:37:52.291249 7023 raft_consensus.cc:2454] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 149202ea7b1341ccb41b07b9e36d1752 in term 1.
I20260620 11:37:52.291414 7096 leader_election.cc:302] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149202ea7b1341ccb41b07b9e36d1752, f379b6ea23a04ffe9a852e214e6f7998; no voters:
I20260620 11:37:52.291543 7240 raft_consensus.cc:2781] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:52.291630 7240 raft_consensus.cc:686] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 1 LEADER]: Becoming Leader. State: Replica: 149202ea7b1341ccb41b07b9e36d1752, State: Running, Role: LEADER
I20260620 11:37:52.291687 7240 consensus_queue.cc:237] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.292300 6725 catalog_manager.cc:5495] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 reported cstate change: term changed from 0 to 1, leader changed from <none> to 149202ea7b1341ccb41b07b9e36d1752 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "149202ea7b1341ccb41b07b9e36d1752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } health_report { overall_health: HEALTHY } } }
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[6681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955471, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:52.322907 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.314792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41393 (local address 127.0.0.1:44074)
0620 11:37:52.314999 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.315008 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.315070 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.315267 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.315268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.315583 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:52.315632 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.315634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.316654 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.316655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.317293 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.317295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.317760 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.317761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.317806 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.318459 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:52.318464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:52.320066 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:52.321818 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:52.321826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:52.321828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:52.322105 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:52.322310 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:52.322311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:52.322311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:52.322372 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:52.322627 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:52.322634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:52.322676 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.322727 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.322831 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":102,"threads_started":1}
I20260620 11:37:52.323006 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.314855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44074 (local address 127.5.122.190:41393)
0620 11:37:52.314935 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.314937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.315080 (+ 143us) server_negotiation.cc:407] Connection header received
0620 11:37:52.315118 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.315119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.315131 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.315220 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:52.315751 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.316551 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.317372 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.317670 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.317740 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.320168 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:52.320170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:52.320170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:52.320178 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:52.321731 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.322178 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.322179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.322179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.322239 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.322431 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.322432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.322432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.322569 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:52.322630 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.322770 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.322894 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":50}
I20260620 11:37:52.325372 6725 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:44074:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 11:37:52.325629 6725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:52.327086 6863 tablet_service.cc:1467] Processing CreateTablet for tablet ddc25aa5a67d43aca6fd2db8957120eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0954a3804e924dcca53296576e96a1f7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:37:52.327113 7143 tablet_service.cc:1467] Processing CreateTablet for tablet ddc25aa5a67d43aca6fd2db8957120eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0954a3804e924dcca53296576e96a1f7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:37:52.327275 6863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc25aa5a67d43aca6fd2db8957120eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.327283 7143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc25aa5a67d43aca6fd2db8957120eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.328418 7233 tablet_bootstrap.cc:492] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: Bootstrap starting.
I20260620 11:37:52.328863 7003 tablet_service.cc:1467] Processing CreateTablet for tablet ddc25aa5a67d43aca6fd2db8957120eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0954a3804e924dcca53296576e96a1f7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:37:52.328967 7233 tablet_bootstrap.cc:654] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.329052 7003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddc25aa5a67d43aca6fd2db8957120eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:52.329638 7233 tablet_bootstrap.cc:492] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: No bootstrap required, opened a new log
I20260620 11:37:52.329689 7233 ts_tablet_manager.cc:1397] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:52.329847 7233 raft_consensus.cc:348] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.329901 7233 raft_consensus.cc:374] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.329916 7233 raft_consensus.cc:729] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b81650d705441d9b366ae7607b55a7, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.329996 7233 consensus_queue.cc:260] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.330036 7232 tablet_bootstrap.cc:492] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: Bootstrap starting.
I20260620 11:37:52.330096 7233 ts_tablet_manager.cc:1428] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:52.330209 7238 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.330300 7238 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:52.330355 7232 tablet_bootstrap.cc:654] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.331501 7234 tablet_bootstrap.cc:492] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: Bootstrap starting.
I20260620 11:37:52.331728 7232 tablet_bootstrap.cc:492] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: No bootstrap required, opened a new log
I20260620 11:37:52.331772 7232 ts_tablet_manager.cc:1397] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:52.331902 7234 tablet_bootstrap.cc:654] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:52.331885 7232 raft_consensus.cc:348] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.331950 7232 raft_consensus.cc:374] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.331971 7232 raft_consensus.cc:729] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149202ea7b1341ccb41b07b9e36d1752, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.332015 7232 consensus_queue.cc:260] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.332108 7232 ts_tablet_manager.cc:1428] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:52.332165 7240 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.332278 7240 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:52.332823 7234 tablet_bootstrap.cc:492] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: No bootstrap required, opened a new log
I20260620 11:37:52.332876 7234 ts_tablet_manager.cc:1397] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:52.333031 7234 raft_consensus.cc:348] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.333096 7234 raft_consensus.cc:374] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:52.333117 7234 raft_consensus.cc:729] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f379b6ea23a04ffe9a852e214e6f7998, State: Initialized, Role: FOLLOWER
I20260620 11:37:52.333176 7234 consensus_queue.cc:260] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.333273 7234 ts_tablet_manager.cc:1428] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:52.333395 7239 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.333492 7239 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: This TxnStatusTablet replica's current role is: FOLLOWER
W20260620 11:37:52.360473 7210 tablet.cc:2369] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:37:52.402585 7239 raft_consensus.cc:482] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:52.402715 7239 raft_consensus.cc:504] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.403167 7239 leader_election.cc:288] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239), 149202ea7b1341ccb41b07b9e36d1752 (127.5.122.131:45363)
I20260620 11:37:52.407240 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.403492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:46435 (local address 127.5.122.129:46239)
0620 11:37:52.403545 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.403548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.403709 (+ 161us) server_negotiation.cc:407] Connection header received
0620 11:37:52.403755 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.403756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.403766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.403823 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:52.404019 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.404741 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.406434 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.406801 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.406901 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.406912 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.407101 (+ 189us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.407155 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:37:52.407255 6962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.403401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:45363 (local address 127.5.122.130:40549)
0620 11:37:52.403469 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.403484 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.403537 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.403725 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.403726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.403728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:52.403815 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.403817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.404632 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.404633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.406298 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.406302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.406928 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.406929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.407024 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.407027 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.407102 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.407152 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":72}
I20260620 11:37:52.407267 7271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.403411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46239 (local address 127.5.122.130:46435)
0620 11:37:52.403623 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.403634 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.403692 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.403838 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.403838 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.403839 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:52.403943 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.403945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.404825 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.404826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.406299 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.406302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.406914 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.406916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.407014 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.407018 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.407064 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.407141 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":5,"thread_start_us":127,"threads_started":1}
I20260620 11:37:52.407282 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.403499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:40549 (local address 127.5.122.131:45363)
0620 11:37:52.403547 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.403548 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.403560 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:37:52.403597 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.403598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.403612 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.403687 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:52.403900 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.404543 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.406402 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.406793 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.406925 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.406935 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.407123 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.407172 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:37:52.407584 6864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc25aa5a67d43aca6fd2db8957120eb" candidate_uuid: "f379b6ea23a04ffe9a852e214e6f7998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b81650d705441d9b366ae7607b55a7" is_pre_election: true
I20260620 11:37:52.407573 7163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc25aa5a67d43aca6fd2db8957120eb" candidate_uuid: "f379b6ea23a04ffe9a852e214e6f7998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149202ea7b1341ccb41b07b9e36d1752" is_pre_election: true
I20260620 11:37:52.407713 6864 raft_consensus.cc:2454] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f379b6ea23a04ffe9a852e214e6f7998 in term 0.
I20260620 11:37:52.407738 7163 raft_consensus.cc:2454] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f379b6ea23a04ffe9a852e214e6f7998 in term 0.
I20260620 11:37:52.407915 6956 leader_election.cc:302] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e3b81650d705441d9b366ae7607b55a7, f379b6ea23a04ffe9a852e214e6f7998; no voters:
I20260620 11:37:52.408092 7239 raft_consensus.cc:2781] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:52.408161 7239 raft_consensus.cc:482] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:52.408185 7239 raft_consensus.cc:3037] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.408900 7239 raft_consensus.cc:504] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.409080 7239 leader_election.cc:288] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [CANDIDATE]: Term 1 election: Requested vote from peers e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239), 149202ea7b1341ccb41b07b9e36d1752 (127.5.122.131:45363)
I20260620 11:37:52.409243 7163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc25aa5a67d43aca6fd2db8957120eb" candidate_uuid: "f379b6ea23a04ffe9a852e214e6f7998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149202ea7b1341ccb41b07b9e36d1752"
I20260620 11:37:52.409327 7163 raft_consensus.cc:3037] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.409241 6864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddc25aa5a67d43aca6fd2db8957120eb" candidate_uuid: "f379b6ea23a04ffe9a852e214e6f7998" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3b81650d705441d9b366ae7607b55a7"
I20260620 11:37:52.409355 6864 raft_consensus.cc:3037] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:52.410089 7163 raft_consensus.cc:2454] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f379b6ea23a04ffe9a852e214e6f7998 in term 1.
I20260620 11:37:52.410153 6864 raft_consensus.cc:2454] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f379b6ea23a04ffe9a852e214e6f7998 in term 1.
I20260620 11:37:52.410318 6955 leader_election.cc:302] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 149202ea7b1341ccb41b07b9e36d1752, f379b6ea23a04ffe9a852e214e6f7998; no voters:
I20260620 11:37:52.410439 7239 raft_consensus.cc:2781] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:52.410596 7239 raft_consensus.cc:686] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [term 1 LEADER]: Becoming Leader. State: Replica: f379b6ea23a04ffe9a852e214e6f7998, State: Running, Role: LEADER
I20260620 11:37:52.410686 7239 consensus_queue.cc:237] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } }
I20260620 11:37:52.411079 7272 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: TxnStatusTablet state changed. Reason: New leader f379b6ea23a04ffe9a852e214e6f7998. Latest consensus state: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.411175 7272 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:37:52.411461 7274 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 11:37:52.411581 7274 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 11:37:52.411609 6725 catalog_manager.cc:5495] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 reported cstate change: term changed from 0 to 1, leader changed from <none> to f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } health_report { overall_health: UNKNOWN } } }
I20260620 11:37:52.428604 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.425633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:37473 (local address 127.0.0.1:54958)
0620 11:37:52.425718 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.425733 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.425791 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.426032 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.426033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.426034 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:37:52.426099 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.426101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.427016 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.427017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.427597 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.427599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.428077 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.428078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.428139 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.428156 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:37:52.428456 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:37:52.428463 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.428500 (+ 37us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.428541 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 11:37:52.428615 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.425751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54958 (local address 127.5.122.130:37473)
0620 11:37:52.425830 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.425832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.425843 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:37:52.425880 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.425881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.425903 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.425996 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:52.426169 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.426933 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.427687 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.428000 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.428071 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.428251 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:52.428389 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:52.428466 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.428504 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.428526 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 11:37:52.429908 6864 raft_consensus.cc:1261] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Refusing update from remote peer f379b6ea23a04ffe9a852e214e6f7998: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:52.430002 7163 raft_consensus.cc:1261] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Refusing update from remote peer f379b6ea23a04ffe9a852e214e6f7998: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:52.430338 7239 consensus_queue.cc:1035] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:52.430502 7272 consensus_queue.cc:1035] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998 [LEADER]: Connected to new peer: Peer: permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:52.431147 7238 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: TxnStatusTablet state changed. Reason: New leader f379b6ea23a04ffe9a852e214e6f7998. Latest consensus state: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.431247 7238 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:52.431246 7240 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: TxnStatusTablet state changed. Reason: New leader f379b6ea23a04ffe9a852e214e6f7998. Latest consensus state: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.431342 7240 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:52.432060 7272 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.432154 7272 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:37:52.432570 7246 mvcc.cc:205] Tried to move back new op lower bound from 7298889615070597120 to 7298889614995034112. Current Snapshot: MvccSnapshot[applied={T|T < 7298889615070597120}]
I20260620 11:37:52.432644 7238 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.432729 7238 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P e3b81650d705441d9b366ae7607b55a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:52.433535 7239 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.433640 7239 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P f379b6ea23a04ffe9a852e214e6f7998: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:37:52.434176 7245 mvcc.cc:205] Tried to move back new op lower bound from 7298889615070597120 to 7298889614995034112. Current Snapshot: MvccSnapshot[applied={T|T < 7298889615070597120}]
I20260620 11:37:52.434335 7240 tablet_replica.cc:439] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f379b6ea23a04ffe9a852e214e6f7998" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 } } peers { permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 } } peers { permanent_uuid: "149202ea7b1341ccb41b07b9e36d1752" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 45363 } } }
I20260620 11:37:52.434417 7240 tablet_replica.cc:442] T ddc25aa5a67d43aca6fd2db8957120eb P 149202ea7b1341ccb41b07b9e36d1752: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:52.439054 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.436222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:37473 (local address 127.5.122.131:52311)
0620 11:37:52.436309 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.436323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.436385 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.436506 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.436507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.436508 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:37:52.436566 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.436568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.437363 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.437364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.438145 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.438148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.438603 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.438604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.438655 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.438673 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:37:52.438878 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:37:52.438880 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.438933 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.438984 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 11:37:52.439042 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.436297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:52311 (local address 127.5.122.130:37473)
0620 11:37:52.436340 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.436343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.436366 (+ 23us) server_negotiation.cc:407] Connection header received
0620 11:37:52.436434 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.436435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.436447 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.436502 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:52.436633 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.437276 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.438257 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.438497 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.438560 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.438781 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:52.438818 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:52.438869 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.438935 (+ 66us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.438958 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:37:52.442160 6864 raft_consensus.cc:1261] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Refusing update from remote peer 149202ea7b1341ccb41b07b9e36d1752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:52.442162 7023 raft_consensus.cc:1261] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Refusing update from remote peer 149202ea7b1341ccb41b07b9e36d1752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:52.442476 7250 consensus_queue.cc:1035] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3b81650d705441d9b366ae7607b55a7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:52.442606 7240 consensus_queue.cc:1035] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f379b6ea23a04ffe9a852e214e6f7998" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 37473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:52.452514 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.449676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:53871 (local address 127.5.122.131:45363)
0620 11:37:52.449720 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.449722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.449731 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:37:52.449798 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.449799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.449809 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.449875 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:52.450054 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.450582 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.451763 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.451964 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.452020 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.452245 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:52.452304 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:52.452343 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.452423 (+ 80us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.452444 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:37:52.452580 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.449482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:45363 (local address 127.5.122.130:53871)
0620 11:37:52.449645 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:52.449658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:52.449721 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:52.449900 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:52.449901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:52.449902 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:37:52.449999 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.450001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.450665 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.450667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.451631 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:52.451634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.452067 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.452068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:52.452138 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.452159 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:37:52.452364 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:37:52.452365 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 11:37:52.452421 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 11:37:52.452475 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260620 11:37:52.460978 6726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:52.461108 6726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:52.461791 6726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d696e08d79d844c9bb59485b562fc5ec: Sending DeleteTablet for 3 replicas of tablet c5192aade1ba4c7fbcff1ca46e8ea84a
I20260620 11:37:52.462052 6863 tablet_service.cc:1514] Processing DeleteTablet for tablet c5192aade1ba4c7fbcff1ca46e8ea84a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:52 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:57752
I20260620 11:37:52.462071 7143 tablet_service.cc:1514] Processing DeleteTablet for tablet c5192aade1ba4c7fbcff1ca46e8ea84a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:52 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58684
I20260620 11:37:52.462165 7255 tablet_replica.cc:330] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: stopping tablet replica
I20260620 11:37:52.462167 7257 tablet_replica.cc:330] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: stopping tablet replica
I20260620 11:37:52.462213 7255 raft_consensus.cc:2229] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:52.462246 7255 raft_consensus.cc:2258] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:52.462239 7257 raft_consensus.cc:2229] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:37:52.462296 7257 raft_consensus.cc:2258] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:52.462435 7003 tablet_service.cc:1514] Processing DeleteTablet for tablet c5192aade1ba4c7fbcff1ca46e8ea84a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:52 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:54946
I20260620 11:37:52.462524 7256 tablet_replica.cc:330] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: stopping tablet replica
I20260620 11:37:52.462569 7256 raft_consensus.cc:2229] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:52.462602 7256 raft_consensus.cc:2258] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:52.462638 7255 ts_tablet_manager.cc:1905] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:52.462636 7257 ts_tablet_manager.cc:1905] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:52.462888 7256 ts_tablet_manager.cc:1905] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:52.464306 7257 ts_tablet_manager.cc:1918] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:52.464363 7257 log.cc:1196] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/c5192aade1ba4c7fbcff1ca46e8ea84a
I20260620 11:37:52.464404 7255 ts_tablet_manager.cc:1918] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:52.464427 7256 ts_tablet_manager.cc:1918] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:52.464458 7255 log.cc:1196] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/c5192aade1ba4c7fbcff1ca46e8ea84a
I20260620 11:37:52.464473 7256 log.cc:1196] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/c5192aade1ba4c7fbcff1ca46e8ea84a
I20260620 11:37:52.464588 7257 ts_tablet_manager.cc:1939] T c5192aade1ba4c7fbcff1ca46e8ea84a P 149202ea7b1341ccb41b07b9e36d1752: Deleting consensus metadata
I20260620 11:37:52.464685 7255 ts_tablet_manager.cc:1939] T c5192aade1ba4c7fbcff1ca46e8ea84a P e3b81650d705441d9b366ae7607b55a7: Deleting consensus metadata
I20260620 11:37:52.464685 7256 ts_tablet_manager.cc:1939] T c5192aade1ba4c7fbcff1ca46e8ea84a P f379b6ea23a04ffe9a852e214e6f7998: Deleting consensus metadata
I20260620 11:37:52.464907 6711 catalog_manager.cc:4841] TS 149202ea7b1341ccb41b07b9e36d1752 (127.5.122.131:45363): tablet c5192aade1ba4c7fbcff1ca46e8ea84a (table test-table [id=a02cdde9173448adb3c026ce6c546fe4]) successfully deleted
I20260620 11:37:52.465303 6712 catalog_manager.cc:4841] TS f379b6ea23a04ffe9a852e214e6f7998 (127.5.122.130:37473): tablet c5192aade1ba4c7fbcff1ca46e8ea84a (table test-table [id=a02cdde9173448adb3c026ce6c546fe4]) successfully deleted
I20260620 11:37:52.465433 6712 catalog_manager.cc:4841] TS e3b81650d705441d9b366ae7607b55a7 (127.5.122.129:46239): tablet c5192aade1ba4c7fbcff1ca46e8ea84a (table test-table [id=a02cdde9173448adb3c026ce6c546fe4]) successfully deleted
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[6681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:37:52.475310 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.466240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.5.122.129:46239)
0620 11:37:52.466304 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.466307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.466505 (+ 198us) server_negotiation.cc:407] Connection header received
0620 11:37:52.466556 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.466557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.466569 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.466628 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:52.467158 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.467916 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.468621 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.468927 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.469000 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.472129 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:52.472143 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:52.472143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:52.472152 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:52.474133 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.474492 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.474498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.474498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.474569 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.474752 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.474752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.474753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.474948 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:52.475106 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.475141 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.475220 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260620 11:37:52.475677 6923 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57760
I20260620 11:37:52.482813 6775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:52.476714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.5.122.190:41393)
0620 11:37:52.476778 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:52.476780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:52.476790 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:52.476818 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:52.476819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:52.476830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:52.476886 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:52.477417 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.477931 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.478685 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:52.478905 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:52.478982 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:52.480190 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:52.480192 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:52.480193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:52.480201 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:52.481686 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.482118 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.482118 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.482119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.482170 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:52.482364 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:52.482365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:52.482365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:52.482463 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:52.482568 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:52.482682 (+ 114us) server_negotiation.cc:299] Negotiation successful
0620 11:37:52.482743 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260620 11:37:52.483121 6726 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44076
I20260620 11:37:52.483865 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 6798
I20260620 11:37:52.492597 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 6938
I20260620 11:37:52.498528 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 7078
I20260620 11:37:52.504182 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 6694
2026-06-20T11:37:52Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2217 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[7293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[7293](info): set up 2 sockets
Jun 20 11:37:52 dist-test-slave-szdr krb5kdc[7293](info): commencing operation
krb5kdc: starting...
W20260620 11:37:53.621601 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:37:53 dist-test-slave-szdr krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:37:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:37:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:37:53.744606 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37155
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:37155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:53.828398 7306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:53.828646 7306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:53.828671 7306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:53.830024 7306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:37:53.830081 7306 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 11:37:53.830096 7306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:53.830109 7306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:53.830121 7306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:37:53.830133 7306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:37:53.831780 7306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:37155
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:53.832065 7306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:53.832315 7306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:53.837459 7314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:53.837433 7311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:53.837450 7312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:53.837479 7306 server_base.cc:1034] running on GCE node
I20260620 11:37:53.837877 7306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:53.838198 7306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:53.839371 7306 hybrid_clock.cc:648] HybridClock initialized: now 1781955473839349 us; error 35 us; skew 500 ppm
Jun 20 11:37:53 dist-test-slave-szdr krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955473, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:53.842188 7306 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:37:53.842655 7306 webserver.cc:460] Webserver started at http://127.5.122.190:37897/ using document root <none> and password file <none>
I20260620 11:37:53.842906 7306 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:53.842952 7306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:53.843103 7306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:53.844209 7306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "a46257f10b9f46f18ca2bc1da2956801"
format_stamp: "Formatted at 2026-06-20 11:37:53 on dist-test-slave-szdr"
server_key: "3ac5950e4e2ad1b27dce852622654a5b"
server_key_iv: "db5d0f0c500a3a627a4d589da171a752"
server_key_version: "encryptionkey@0"
I20260620 11:37:53.844585 7306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "a46257f10b9f46f18ca2bc1da2956801"
format_stamp: "Formatted at 2026-06-20 11:37:53 on dist-test-slave-szdr"
server_key: "3ac5950e4e2ad1b27dce852622654a5b"
server_key_iv: "db5d0f0c500a3a627a4d589da171a752"
server_key_version: "encryptionkey@0"
I20260620 11:37:53.846086 7306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:53.846971 7321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:53.847146 7306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:53.847262 7306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "a46257f10b9f46f18ca2bc1da2956801"
format_stamp: "Formatted at 2026-06-20 11:37:53 on dist-test-slave-szdr"
server_key: "3ac5950e4e2ad1b27dce852622654a5b"
server_key_iv: "db5d0f0c500a3a627a4d589da171a752"
server_key_version: "encryptionkey@0"
I20260620 11:37:53.847357 7306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:53.865792 7306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:53.868315 7306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:53.868525 7306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:53.873382 7306 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:37155
I20260620 11:37:53.873430 7383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:37155 every 8 connection(s)
I20260620 11:37:53.873824 7306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:37:53.874619 7384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:53.877094 7384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Bootstrap starting.
I20260620 11:37:53.877810 7384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:53.878158 7384 log.cc:824] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:53.878870 7384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: No bootstrap required, opened a new log
I20260620 11:37:53.879938 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 7306
I20260620 11:37:53.880021 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:37:53.880191 5610 external_mini_cluster.cc:1428] Setting key 10efbf246400fb9857e4af0c084f6071
I20260620 11:37:53.881455 7384 raft_consensus.cc:348] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46257f10b9f46f18ca2bc1da2956801" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37155 } }
I20260620 11:37:53.881647 7384 raft_consensus.cc:374] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:53.881687 7384 raft_consensus.cc:729] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a46257f10b9f46f18ca2bc1da2956801, State: Initialized, Role: FOLLOWER
I20260620 11:37:53.881820 7384 consensus_queue.cc:260] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46257f10b9f46f18ca2bc1da2956801" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37155 } }
I20260620 11:37:53.881897 7384 raft_consensus.cc:388] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:37:53.881920 7384 raft_consensus.cc:482] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:37:53.882014 7384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:53.882669 7384 raft_consensus.cc:504] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46257f10b9f46f18ca2bc1da2956801" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37155 } }
I20260620 11:37:53.882777 7384 leader_election.cc:302] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a46257f10b9f46f18ca2bc1da2956801; no voters:
I20260620 11:37:53.882932 7384 leader_election.cc:288] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:37:53.883000 7389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:53.883178 7384 sys_catalog.cc:564] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:37:53.883198 7389 raft_consensus.cc:686] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [term 1 LEADER]: Becoming Leader. State: Replica: a46257f10b9f46f18ca2bc1da2956801, State: Running, Role: LEADER
I20260620 11:37:53.883280 7389 consensus_queue.cc:237] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46257f10b9f46f18ca2bc1da2956801" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37155 } }
I20260620 11:37:53.883625 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a46257f10b9f46f18ca2bc1da2956801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46257f10b9f46f18ca2bc1da2956801" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37155 } } }
I20260620 11:37:53.883720 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:53.883615 7389 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a46257f10b9f46f18ca2bc1da2956801. Latest consensus state: current_term: 1 leader_uuid: "a46257f10b9f46f18ca2bc1da2956801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46257f10b9f46f18ca2bc1da2956801" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37155 } } }
I20260620 11:37:53.883899 7389 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801 [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:53.884016 7398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:37:53.884526 7398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:37:53.886250 7398 catalog_manager.cc:1347] Generated new cluster ID: 1f63b7cfbf234c5a9cbd6d7c6c29101e
I20260620 11:37:53.886307 7398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:37:53 dist-test-slave-szdr krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:53.891721 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:53.880791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32910 (local address 127.5.122.190:37155)
0620 11:37:53.881043 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:53.881055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:53.881098 (+ 43us) server_negotiation.cc:407] Connection header received
0620 11:37:53.881414 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:53.881423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:53.881491 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:53.881660 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:53.882447 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:53.883932 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:53.885212 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:53.885843 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:53.885995 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:53.888314 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:53.888317 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:53.888318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:53.888341 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:53.890151 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:53.890533 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:53.890536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:53.890536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:53.890643 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:53.890866 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:53.890867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:53.890868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:53.891110 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:53.891282 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:53.891379 (+ 97us) server_negotiation.cc:299] Negotiation successful
0620 11:37:53.891534 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260620 11:37:53.897800 7398 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:37:53.898550 7398 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:37:53.905886 7398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Generated new TSK 0
I20260620 11:37:53.906149 7398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:37:53.936707 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37155
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:54.022744 7410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:54.023021 7410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:54.023072 7410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:54.024374 7410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:54.024457 7410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:54.024501 7410 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 11:37:54.024529 7410 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:37:54.024564 7410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:37:54.026283 7410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:37155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:54.026587 7410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:54.026839 7410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:54.032232 7415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:54.032226 7416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:54.032420 7410 server_base.cc:1034] running on GCE node
W20260620 11:37:54.032224 7418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:54.032702 7410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:54.032994 7410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:54.034173 7410 hybrid_clock.cc:648] HybridClock initialized: now 1781955474034158 us; error 30 us; skew 500 ppm
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:54.036860 7410 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:37:54.037400 7410 webserver.cc:460] Webserver started at http://127.5.122.129:34835/ using document root <none> and password file <none>
I20260620 11:37:54.037647 7410 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:54.037719 7410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:54.037880 7410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:54.039031 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "49b677342267486e8fe59d12d1905f28"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "dc5b98046eb34d3d2d0a70398fd5a940"
server_key_iv: "806846871c351213671ef49d453250e1"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.039415 7410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "49b677342267486e8fe59d12d1905f28"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "dc5b98046eb34d3d2d0a70398fd5a940"
server_key_iv: "806846871c351213671ef49d453250e1"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.040820 7410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:54.041590 7425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.041867 7410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:54.042003 7410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "49b677342267486e8fe59d12d1905f28"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "dc5b98046eb34d3d2d0a70398fd5a940"
server_key_iv: "806846871c351213671ef49d453250e1"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.042093 7410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:54.060256 7410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:54.062430 7410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:54.062613 7410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:54.063230 7410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:54.063292 7410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.063338 7410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:54.063365 7410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:54.071019 7410 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:32981
I20260620 11:37:54.071440 7410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:37:54.071936 7540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:32981 every 8 connection(s)
I20260620 11:37:54.072378 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 7410
I20260620 11:37:54.072495 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:37:54.072685 5610 external_mini_cluster.cc:1428] Setting key f671b22e4499671707205a13a5ff836a
I20260620 11:37:54.074580 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.063700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:42095 (local address 127.5.122.190:37155)
0620 11:37:54.063754 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.063756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.063959 (+ 203us) server_negotiation.cc:407] Connection header received
0620 11:37:54.064518 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.064519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.064531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.064566 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.065444 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.066058 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.067158 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.067381 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.067463 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.071037 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.071039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.071040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.071053 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.072988 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.073500 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.073501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.073501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.073568 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.073839 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.073840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.073841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.074017 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.074084 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.074385 (+ 301us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.074487 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:37:54.074800 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.063688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37155 (local address 127.5.122.129:42095)
0620 11:37:54.063836 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.063872 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.064396 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.064655 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.064659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.064989 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:54.065343 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.065349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.066153 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.066154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.067045 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.067048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.067468 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.067469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.067683 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.068362 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:54.068375 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:54.070916 (+ 2541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:54.073096 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.073103 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.073105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.073414 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.073646 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.073647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.073647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.073766 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.074081 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:54.074089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:54.074242 (+ 153us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.074352 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.074526 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260620 11:37:54.075352 7541 heartbeater.cc:347] Connected to a master server at 127.5.122.190:37155
I20260620 11:37:54.075501 7541 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:54.075771 7541 heartbeater.cc:511] Master 127.5.122.190:37155 requested a full tablet report, sending...
I20260620 11:37:54.076681 7337 ts_manager.cc:194] Registered new tserver with Master: 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981)
I20260620 11:37:54.077472 7337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:42095
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
I20260620 11:37:54.082695 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.076291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37155 (local address 127.5.122.129:48467)
0620 11:37:54.076415 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.076427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.076490 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.076612 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.076613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.076820 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:54.076896 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.076898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.077905 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.077906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.078524 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.078526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.078934 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.078935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.079011 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.079561 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:54.079566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:54.079876 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:54.081623 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.081624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.081625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.081844 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.082100 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.082100 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.082101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.082154 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.082412 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:54.082413 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:54.082467 (+ 54us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.082520 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.082612 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260620 11:37:54.082697 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.076354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:48467 (local address 127.5.122.190:37155)
0620 11:37:54.076395 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.076397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.076492 (+ 95us) server_negotiation.cc:407] Connection header received
0620 11:37:54.076538 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.076539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.076549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.076607 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.076977 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.077801 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.078623 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.078844 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.078932 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.079974 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.079975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.079976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.079983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.081476 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.081929 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.081930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.081931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.082025 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.082218 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.082219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.082220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.082350 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.082406 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.082553 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.082633 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:37:54.101752 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37155
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:54.187461 7550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:54.187696 7550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:54.187726 7550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:54.189040 7550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:54.189095 7550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:54.189119 7550 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 11:37:54.189163 7550 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:37:54.189199 7550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:37:54.190877 7550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:37155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:54.191174 7550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:54.191434 7550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:54.197149 7556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:54.197198 7555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:54.197152 7558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:54.197449 7550 server_base.cc:1034] running on GCE node
I20260620 11:37:54.197645 7550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:54.197877 7550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:54.199031 7550 hybrid_clock.cc:648] HybridClock initialized: now 1781955474199026 us; error 41 us; skew 500 ppm
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:54.201666 7550 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:37:54.202210 7550 webserver.cc:460] Webserver started at http://127.5.122.130:45925/ using document root <none> and password file <none>
I20260620 11:37:54.202412 7550 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:54.202454 7550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:54.202548 7550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:54.203626 7550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "055f36cc7ef24940b3515ae00577f32b"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "2101c4a6afd0fd7e02e006d7710d3c0a"
server_key_iv: "ffcd5fb5b443a296962cddc749d141df"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.203943 7550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "055f36cc7ef24940b3515ae00577f32b"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "2101c4a6afd0fd7e02e006d7710d3c0a"
server_key_iv: "ffcd5fb5b443a296962cddc749d141df"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.205331 7550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:54.206137 7565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.206368 7550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:54.206482 7550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "055f36cc7ef24940b3515ae00577f32b"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "2101c4a6afd0fd7e02e006d7710d3c0a"
server_key_iv: "ffcd5fb5b443a296962cddc749d141df"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.206578 7550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:54.217376 7550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:54.219697 7550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:54.219890 7550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:54.220448 7550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:54.220518 7550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.220564 7550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:54.220598 7550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.227911 7550 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:45065
I20260620 11:37:54.228008 7680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:45065 every 8 connection(s)
I20260620 11:37:54.228444 7550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:54.232461 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.221133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:45263 (local address 127.5.122.190:37155)
0620 11:37:54.221181 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.221184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.221384 (+ 200us) server_negotiation.cc:407] Connection header received
0620 11:37:54.221979 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.221981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.222005 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.222064 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.223104 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.223977 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.225782 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.226141 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.226201 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.229373 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.229375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.229375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.229383 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.230978 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.231429 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.231431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.231431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.231497 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.231770 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.231771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.231772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.231895 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.231967 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.232282 (+ 315us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.232367 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:37:54.232667 7574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.221036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37155 (local address 127.5.122.130:45263)
0620 11:37:54.221244 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.221291 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.221833 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.222169 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.222174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.222622 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:54.222955 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.222962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.224098 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.224099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.225654 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.225657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.226234 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.226236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.226411 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.227137 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:54.227149 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:54.229258 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:54.231098 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.231106 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.231109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.231347 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.231573 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.231574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.231574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.231694 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.231971 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:54.231978 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:54.232133 (+ 155us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.232245 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.232422 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260620 11:37:54.233242 7681 heartbeater.cc:347] Connected to a master server at 127.5.122.190:37155
I20260620 11:37:54.233337 7681 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:54.233541 7681 heartbeater.cc:511] Master 127.5.122.190:37155 requested a full tablet report, sending...
I20260620 11:37:54.233938 7338 ts_manager.cc:194] Registered new tserver with Master: 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065)
I20260620 11:37:54.234499 7338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:45263
I20260620 11:37:54.237022 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 7550
I20260620 11:37:54.237128 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:37:54.237323 5610 external_mini_cluster.cc:1428] Setting key 0b2bee8c85fad75428ca2cfd5b271620
I20260620 11:37:54.240458 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.234440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:45385 (local address 127.5.122.190:37155)
0620 11:37:54.234484 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.234486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.234553 (+ 67us) server_negotiation.cc:407] Connection header received
0620 11:37:54.234701 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.234701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.234709 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.234765 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.235115 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.235608 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.236325 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.236524 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.236617 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.237654 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.237656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.237657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.237664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.239234 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.239601 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.239602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.239603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.239666 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.239874 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.239875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.239876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.240037 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.240114 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.240280 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.240362 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:37:54.240473 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.234368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37155 (local address 127.5.122.130:45385)
0620 11:37:54.234497 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.234509 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.234570 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.234782 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.234783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.234976 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:54.235029 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.235031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.235708 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.235709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.236236 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.236237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.236626 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.236628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.236704 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.237213 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:54.237217 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:54.237566 (+ 349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:54.239325 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.239326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.239326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.239505 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.239738 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.239739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.239740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.239812 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.240116 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:54.240117 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:54.240166 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.240253 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.240364 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:37:54.265935 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37155
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:43743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:54.352720 7690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:54.352965 7690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:54.352995 7690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:54.354259 7690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:54.354302 7690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:54.354324 7690 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 11:37:54.354336 7690 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:37:54.354354 7690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:37:54.355845 7690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:37155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:54.356068 7690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:54.356277 7690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:54.361853 7698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:54.361869 7696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:54.361848 7695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:54.362207 7690 server_base.cc:1034] running on GCE node
I20260620 11:37:54.362407 7690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:54.362634 7690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:54.363801 7690 hybrid_clock.cc:648] HybridClock initialized: now 1781955474363791 us; error 34 us; skew 500 ppm
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:54.366554 7690 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:37:54.367029 7690 webserver.cc:460] Webserver started at http://127.5.122.131:37791/ using document root <none> and password file <none>
I20260620 11:37:54.367266 7690 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:54.367314 7690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:54.367465 7690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:54.368588 7690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "d1263692f7d44745a587333cd534fdd7"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "1839289e29f38db54bce934132b1fc22"
server_key_iv: "643e53941290506c7f1af6ca544754a1"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.368973 7690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "d1263692f7d44745a587333cd534fdd7"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "1839289e29f38db54bce934132b1fc22"
server_key_iv: "643e53941290506c7f1af6ca544754a1"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.370589 7690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:54.371423 7705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.371764 7690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:54.371868 7690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "d1263692f7d44745a587333cd534fdd7"
format_stamp: "Formatted at 2026-06-20 11:37:54 on dist-test-slave-szdr"
server_key: "1839289e29f38db54bce934132b1fc22"
server_key_iv: "643e53941290506c7f1af6ca544754a1"
server_key_version: "encryptionkey@0"
I20260620 11:37:54.371943 7690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:54.395771 7690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:54.398204 7690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:54.398375 7690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:54.398937 7690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:37:54.399008 7690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.399055 7690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:37:54.399082 7690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:54.406452 7690 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:41235
I20260620 11:37:54.406636 7820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:41235 every 8 connection(s)
I20260620 11:37:54.406920 7690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:37:54.409031 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.399577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:56585 (local address 127.5.122.190:37155)
0620 11:37:54.399657 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.399660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.399851 (+ 191us) server_negotiation.cc:407] Connection header received
0620 11:37:54.400411 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.400413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.400426 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.400474 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.401361 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.402280 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.403392 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.403611 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.403688 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.406034 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.406036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.406036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.406045 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.407694 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.408145 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.408146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.408146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.408204 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.408443 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.408444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.408445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.408591 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.408657 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.408890 (+ 233us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.408967 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 11:37:54.409261 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.399529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37155 (local address 127.5.122.131:56585)
0620 11:37:54.399721 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.399766 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.400303 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.400550 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.400554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.400922 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:54.401244 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.401251 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.402388 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.402390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.403297 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.403299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.403714 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.403715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.403849 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.404382 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:54.404391 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:54.405902 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:54.407767 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.407771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.407772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.408073 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.408259 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.408259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.408260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.408358 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.408653 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:54.408658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:54.408775 (+ 117us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.408859 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.409035 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260620 11:37:54.409756 7821 heartbeater.cc:347] Connected to a master server at 127.5.122.190:37155
I20260620 11:37:54.409861 7821 heartbeater.cc:464] Registering TS with master...
I20260620 11:37:54.410079 7821 heartbeater.cc:511] Master 127.5.122.190:37155 requested a full tablet report, sending...
I20260620 11:37:54.410459 7338 ts_manager.cc:194] Registered new tserver with Master: d1263692f7d44745a587333cd534fdd7 (127.5.122.131:41235)
I20260620 11:37:54.410967 7338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:56585
I20260620 11:37:54.411453 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 7690
I20260620 11:37:54.411545 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:37:54.411715 5610 external_mini_cluster.cc:1428] Setting key 321302b403d9a79f61e4b96b189bd608
I20260620 11:37:54.412539 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:37:54.416873 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.410813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37155 (local address 127.5.122.131:46285)
0620 11:37:54.410969 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.410986 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.411057 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.411238 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.411239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.411420 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:54.411484 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.411486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.412357 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.412357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.412977 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.412979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.413345 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.413346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.413424 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.413891 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:54.413896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:54.414243 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:54.415840 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.415841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.415841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.416016 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.416221 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.416222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.416222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.416281 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.416532 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:54.416533 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:54.416580 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.416662 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.416772 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260620 11:37:54.416867 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.410874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:46285 (local address 127.5.122.190:37155)
0620 11:37:54.410916 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.410918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.411043 (+ 125us) server_negotiation.cc:407] Connection header received
0620 11:37:54.411160 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.411161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.411171 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.411232 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.411590 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.412269 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.413058 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.413266 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.413336 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.414345 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.414346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.414347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.414353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.415744 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.416096 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.416097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.416097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.416159 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.416348 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.416349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.416349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.416469 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.416541 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.416687 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.416770 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:54.436160 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.428911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.5.122.190:37155)
0620 11:37:54.428986 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.428989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.428999 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:54.429033 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.429034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.429043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.429098 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.429699 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.430248 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.431022 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.431221 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.431297 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.433657 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.433658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.433659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.433690 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.435019 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.435389 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.435391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.435391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.435446 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.435638 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.435639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.435639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.435754 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.435857 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.435965 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.436052 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:37:54.437723 7338 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:37:54.439394 7338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:54.446313 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.441808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.5.122.131:41235)
0620 11:37:54.442364 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.442368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.442378 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:54.442433 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.442435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.442451 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.442720 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:54.442827 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.443949 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.445002 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.445774 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.445878 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.445940 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.446167 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.446207 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":87,"threads_started":1}
I20260620 11:37:54.446319 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.441690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:41235 (local address 127.0.0.1:59778)
0620 11:37:54.441862 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.441907 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.442031 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.442597 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.442598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.442600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:54.442646 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.442648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.444024 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.444025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.444905 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.444908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.445874 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.445875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.445944 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.446026 (+ 82us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.446144 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.446218 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":62,"threads_started":1}
I20260620 11:37:54.446318 7839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.441697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:45065 (local address 127.0.0.1:49734)
0620 11:37:54.442017 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.442029 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.442093 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.442281 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.442284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.442286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:54.442371 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.442374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.443831 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.443832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.445153 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.445156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.446043 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.446043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.446100 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.446103 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.446155 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.446218 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":72,"thread_start_us":73,"threads_started":1}
I20260620 11:37:54.446625 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.441773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.5.122.130:45065)
0620 11:37:54.441994 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.441999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.442082 (+ 83us) server_negotiation.cc:407] Connection header received
0620 11:37:54.442160 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.442163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.442182 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.442260 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:54.442452 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.443720 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.445240 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.445980 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.446049 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.446087 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.446347 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.446538 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260620 11:37:54.447306 7615 tablet_service.cc:1467] Processing CreateTablet for tablet 793aa7b13f2d44689750dd1665e5ee4f (DEFAULT_TABLE table=test-table [id=7003db0134b74a31892200998df9c1ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:54.447376 7755 tablet_service.cc:1467] Processing CreateTablet for tablet 793aa7b13f2d44689750dd1665e5ee4f (DEFAULT_TABLE table=test-table [id=7003db0134b74a31892200998df9c1ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:54.447690 7615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793aa7b13f2d44689750dd1665e5ee4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:54.447732 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793aa7b13f2d44689750dd1665e5ee4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:54.450461 7844 tablet_bootstrap.cc:492] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Bootstrap starting.
I20260620 11:37:54.450816 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.442178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.5.122.129:32981)
0620 11:37:54.442532 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.442536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.442545 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:37:54.442582 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.442585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.442601 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.442685 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:54.442882 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.443700 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.446138 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.446904 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.446994 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.447065 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.450668 (+ 3603us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.450724 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":85,"threads_started":1}
I20260620 11:37:54.450995 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.441896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:32981 (local address 127.0.0.1:58620)
0620 11:37:54.442195 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.442203 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.442243 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.442751 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.442752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.442755 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:54.442794 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.442795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.443778 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.443780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.445153 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.445156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.447007 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.447008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.450109 (+ 3101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.450115 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.450181 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.450234 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":31,"thread_start_us":62,"threads_started":1}
I20260620 11:37:54.450996 7845 tablet_bootstrap.cc:492] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Bootstrap starting.
I20260620 11:37:54.451612 7845 tablet_bootstrap.cc:654] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:54.451894 7844 tablet_bootstrap.cc:654] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:54.451964 7845 log.cc:824] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:54.451833 7475 tablet_service.cc:1467] Processing CreateTablet for tablet 793aa7b13f2d44689750dd1665e5ee4f (DEFAULT_TABLE table=test-table [id=7003db0134b74a31892200998df9c1ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:54.452237 7475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793aa7b13f2d44689750dd1665e5ee4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:54.452239 7844 log.cc:824] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:54.452701 7845 tablet_bootstrap.cc:492] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: No bootstrap required, opened a new log
I20260620 11:37:54.452757 7845 ts_tablet_manager.cc:1397] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:54.454587 7844 tablet_bootstrap.cc:492] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: No bootstrap required, opened a new log
I20260620 11:37:54.454493 7845 raft_consensus.cc:348] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.454658 7844 ts_tablet_manager.cc:1397] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 11:37:54.454667 7845 raft_consensus.cc:374] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:54.454704 7845 raft_consensus.cc:729] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055f36cc7ef24940b3515ae00577f32b, State: Initialized, Role: FOLLOWER
I20260620 11:37:54.454826 7845 consensus_queue.cc:260] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.455108 7845 ts_tablet_manager.cc:1428] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 11:37:54.455181 7681 heartbeater.cc:503] Master 127.5.122.190:37155 was elected leader, sending a full tablet report...
I20260620 11:37:54.455497 7849 tablet_bootstrap.cc:492] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Bootstrap starting.
I20260620 11:37:54.456138 7849 tablet_bootstrap.cc:654] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:54.456472 7849 log.cc:824] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:54.456849 7844 raft_consensus.cc:348] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.457050 7844 raft_consensus.cc:374] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:54.457088 7844 raft_consensus.cc:729] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Initialized, Role: FOLLOWER
I20260620 11:37:54.457237 7849 tablet_bootstrap.cc:492] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: No bootstrap required, opened a new log
I20260620 11:37:54.457208 7844 consensus_queue.cc:260] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.457304 7849 ts_tablet_manager.cc:1397] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:54.457486 7844 ts_tablet_manager.cc:1428] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:37:54.457513 7821 heartbeater.cc:503] Master 127.5.122.190:37155 was elected leader, sending a full tablet report...
I20260620 11:37:54.459148 7849 raft_consensus.cc:348] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.459271 7849 raft_consensus.cc:374] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:54.459293 7849 raft_consensus.cc:729] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b677342267486e8fe59d12d1905f28, State: Initialized, Role: FOLLOWER
I20260620 11:37:54.459367 7849 consensus_queue.cc:260] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.459569 7849 ts_tablet_manager.cc:1428] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 11:37:54.459615 7541 heartbeater.cc:503] Master 127.5.122.190:37155 was elected leader, sending a full tablet report...
W20260620 11:37:54.479177 7682 tablet.cc:2369] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:37:54.572275 7542 tablet.cc:2369] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:37:54.657565 7822 tablet.cc:2369] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:37:54.780405 7851 raft_consensus.cc:482] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:54.780534 7851 raft_consensus.cc:504] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.780844 7851 leader_election.cc:288] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065), 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981)
I20260620 11:37:54.784055 7714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.781064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:45065 (local address 127.5.122.131:50817)
0620 11:37:54.781139 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.781163 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.781233 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.781371 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.781372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.781374 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:54.781467 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.781469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.782253 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.782254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.783206 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.783208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.783778 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.783780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.783884 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.783888 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.783942 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.783978 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":63}
I20260620 11:37:54.784101 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.781135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:50817 (local address 127.5.122.130:45065)
0620 11:37:54.781194 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.781196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.781212 (+ 16us) server_negotiation.cc:407] Connection header received
0620 11:37:54.781293 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.781294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.781304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.781363 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:54.781581 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.782184 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.783293 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.783711 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.783767 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.783774 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.783965 (+ 191us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.784010 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 11:37:54.784193 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.781195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:32981 (local address 127.5.122.131:33757)
0620 11:37:54.781358 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.781367 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.781416 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.781581 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.781582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.781585 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:54.781652 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.781654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.782345 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.782346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.783208 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.783210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.783875 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.783876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.784002 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.784006 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.784065 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.784100 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260620 11:37:54.784205 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.781263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:33757 (local address 127.5.122.129:32981)
0620 11:37:54.781307 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.781309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.781430 (+ 121us) server_negotiation.cc:407] Connection header received
0620 11:37:54.781463 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.781465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.781474 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.781537 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:54.781732 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.782268 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.783297 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.783796 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.783866 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.783873 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.784084 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.784128 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:37:54.784807 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793aa7b13f2d44689750dd1665e5ee4f" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28" is_pre_election: true
I20260620 11:37:54.784844 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793aa7b13f2d44689750dd1665e5ee4f" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b" is_pre_election: true
I20260620 11:37:54.784960 7495 raft_consensus.cc:2454] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:54.784989 7635 raft_consensus.cc:2454] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:54.785184 7706 leader_election.cc:302] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055f36cc7ef24940b3515ae00577f32b, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:54.785296 7851 raft_consensus.cc:2781] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:54.785349 7851 raft_consensus.cc:482] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:54.785384 7851 raft_consensus.cc:3037] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:54.786295 7851 raft_consensus.cc:504] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.786424 7851 leader_election.cc:288] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Requested vote from peers 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065), 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981)
I20260620 11:37:54.786623 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793aa7b13f2d44689750dd1665e5ee4f" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28"
I20260620 11:37:54.786619 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793aa7b13f2d44689750dd1665e5ee4f" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b"
I20260620 11:37:54.786690 7495 raft_consensus.cc:3037] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:54.786708 7635 raft_consensus.cc:3037] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:54.787353 7495 raft_consensus.cc:2454] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:54.787406 7635 raft_consensus.cc:2454] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:54.787532 7706 leader_election.cc:302] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b677342267486e8fe59d12d1905f28, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:54.787622 7851 raft_consensus.cc:2781] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:54.787752 7851 raft_consensus.cc:686] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 1 LEADER]: Becoming Leader. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Running, Role: LEADER
I20260620 11:37:54.787842 7851 consensus_queue.cc:237] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } }
I20260620 11:37:54.788762 7338 catalog_manager.cc:5495] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1263692f7d44745a587333cd534fdd7 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } health_report { overall_health: UNKNOWN } } }
Jun 20 11:37:54 dist-test-slave-szdr krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955473, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:54.817822 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.810817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37155 (local address 127.0.0.1:32928)
0620 11:37:54.810978 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:54.810988 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:54.811090 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:54.811225 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:54.811225 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:54.811566 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:37:54.811621 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.811623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.812277 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.812278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.812805 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:54.812807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.813130 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.813131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:54.813172 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.813676 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:37:54.813680 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:37:54.815119 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:37:54.816742 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.816751 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.816753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.817018 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.817200 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:37:54.817200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:37:54.817201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:37:54.817257 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:37:54.817502 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:37:54.817510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:37:54.817559 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 11:37:54.817626 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 11:37:54.817726 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260620 11:37:54.817915 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:54.810872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32928 (local address 127.5.122.190:37155)
0620 11:37:54.810958 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:54.810961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:54.811085 (+ 124us) server_negotiation.cc:407] Connection header received
0620 11:37:54.811153 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:54.811153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:54.811163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:54.811205 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:54.811712 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.812197 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.812881 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:54.813066 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:54.813116 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:54.815210 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:54.815212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:54.815212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:54.815223 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:54.816643 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.817088 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.817089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.817089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.817141 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:54.817306 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:54.817307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:54.817307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:54.817423 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:54.817479 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:54.817658 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 11:37:54.817772 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":53}
I20260620 11:37:54.820089 7338 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:32928:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 11:37:54.820360 7338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:54.821841 7475 tablet_service.cc:1467] Processing CreateTablet for tablet bebc5bdc5ecb47b580e8ca7c6fa03b72 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a74094b7c7b8493a858dfa476effc6b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:37:54.821866 7615 tablet_service.cc:1467] Processing CreateTablet for tablet bebc5bdc5ecb47b580e8ca7c6fa03b72 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a74094b7c7b8493a858dfa476effc6b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:37:54.821993 7755 tablet_service.cc:1467] Processing CreateTablet for tablet bebc5bdc5ecb47b580e8ca7c6fa03b72 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a74094b7c7b8493a858dfa476effc6b6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:37:54.822062 7475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bebc5bdc5ecb47b580e8ca7c6fa03b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:54.822062 7615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bebc5bdc5ecb47b580e8ca7c6fa03b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:54.822146 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bebc5bdc5ecb47b580e8ca7c6fa03b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:54.823454 7849 tablet_bootstrap.cc:492] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: Bootstrap starting.
I20260620 11:37:54.823668 7845 tablet_bootstrap.cc:492] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: Bootstrap starting.
I20260620 11:37:54.823767 7844 tablet_bootstrap.cc:492] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: Bootstrap starting.
I20260620 11:37:54.823938 7849 tablet_bootstrap.cc:654] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:54.824115 7845 tablet_bootstrap.cc:654] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:54.824172 7844 tablet_bootstrap.cc:654] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:54.824642 7849 tablet_bootstrap.cc:492] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: No bootstrap required, opened a new log
I20260620 11:37:54.824699 7849 ts_tablet_manager.cc:1397] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:54.824900 7849 raft_consensus.cc:348] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:54.824990 7849 raft_consensus.cc:374] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:54.825011 7844 tablet_bootstrap.cc:492] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: No bootstrap required, opened a new log
I20260620 11:37:54.825021 7849 raft_consensus.cc:729] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b677342267486e8fe59d12d1905f28, State: Initialized, Role: FOLLOWER
I20260620 11:37:54.825053 7844 ts_tablet_manager.cc:1397] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:54.825069 7849 consensus_queue.cc:260] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:54.825196 7849 ts_tablet_manager.cc:1428] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:54.825210 7844 raft_consensus.cc:348] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:54.825302 7844 raft_consensus.cc:374] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:54.825333 7844 raft_consensus.cc:729] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Initialized, Role: FOLLOWER
I20260620 11:37:54.825363 7852 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:54.825377 7844 consensus_queue.cc:260] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:54.825443 7852 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:54.825496 7844 ts_tablet_manager.cc:1428] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:54.825610 7851 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:54.825696 7851 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:54.825762 7845 tablet_bootstrap.cc:492] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: No bootstrap required, opened a new log
I20260620 11:37:54.825809 7845 ts_tablet_manager.cc:1397] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:54.826027 7845 raft_consensus.cc:348] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:54.826102 7845 raft_consensus.cc:374] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:54.826117 7845 raft_consensus.cc:729] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055f36cc7ef24940b3515ae00577f32b, State: Initialized, Role: FOLLOWER
I20260620 11:37:54.826174 7845 consensus_queue.cc:260] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:54.826311 7845 ts_tablet_manager.cc:1428] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:54.826357 7848 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:54.826457 7848 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:55.046175 7851 raft_consensus.cc:482] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:55.046295 7851 raft_consensus.cc:504] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.046474 7851 leader_election.cc:288] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981), 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065)
I20260620 11:37:55.046703 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebc5bdc5ecb47b580e8ca7c6fa03b72" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28" is_pre_election: true
I20260620 11:37:55.046803 7495 raft_consensus.cc:2454] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:55.046747 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebc5bdc5ecb47b580e8ca7c6fa03b72" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b" is_pre_election: true
I20260620 11:37:55.046865 7635 raft_consensus.cc:2454] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:55.046994 7706 leader_election.cc:302] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b677342267486e8fe59d12d1905f28, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:55.047106 7851 raft_consensus.cc:2781] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:55.047148 7851 raft_consensus.cc:482] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:55.047175 7851 raft_consensus.cc:3037] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:55.047832 7851 raft_consensus.cc:504] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.047979 7851 leader_election.cc:288] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Requested vote from peers 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981), 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065)
I20260620 11:37:55.048195 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebc5bdc5ecb47b580e8ca7c6fa03b72" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28"
I20260620 11:37:55.048215 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bebc5bdc5ecb47b580e8ca7c6fa03b72" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b"
I20260620 11:37:55.048280 7635 raft_consensus.cc:3037] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:55.048286 7495 raft_consensus.cc:3037] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:55.048915 7495 raft_consensus.cc:2454] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:55.049065 7635 raft_consensus.cc:2454] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:55.049080 7706 leader_election.cc:302] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b677342267486e8fe59d12d1905f28, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:55.049160 7851 raft_consensus.cc:2781] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:55.049216 7851 raft_consensus.cc:686] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [term 1 LEADER]: Becoming Leader. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Running, Role: LEADER
I20260620 11:37:55.049268 7851 consensus_queue.cc:237] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.049602 7854 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: TxnStatusTablet state changed. Reason: New leader d1263692f7d44745a587333cd534fdd7. Latest consensus state: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:55.049693 7854 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:37:55.049862 7866 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 11:37:55.049866 7337 catalog_manager.cc:5495] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1263692f7d44745a587333cd534fdd7 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } health_report { overall_health: HEALTHY } } }
I20260620 11:37:55.050028 7866 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 11:37:55.111634 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.108591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:41235 (local address 127.0.0.1:59788)
0620 11:37:55.108688 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:55.108705 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:55.108770 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:55.109047 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:55.109048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:55.109050 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:37:55.109155 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.109159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.109863 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.109865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.110601 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.110603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.111050 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.111052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.111127 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.111144 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.111439 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.111449 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 11:37:55.111491 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 11:37:55.111560 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 11:37:55.111634 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.108653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.5.122.131:41235)
0620 11:37:55.108813 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.108815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.108825 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:37:55.108862 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.108863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.108948 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.109040 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:55.109235 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.109751 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.110704 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.110967 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.111038 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.111246 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.111366 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.111467 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.111496 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.111525 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
I20260620 11:37:55.113662 7495 raft_consensus.cc:1261] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:55.113682 7635 raft_consensus.cc:1261] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:55.114069 7851 consensus_queue.cc:1035] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:55.114229 7854 consensus_queue.cc:1035] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:55.115538 7852 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: TxnStatusTablet state changed. Reason: New leader d1263692f7d44745a587333cd534fdd7. Latest consensus state: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:55.115651 7852 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:55.116014 7848 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: TxnStatusTablet state changed. Reason: New leader d1263692f7d44745a587333cd534fdd7. Latest consensus state: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:55.116123 7848 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:55.117835 7854 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:55.117941 7854 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:37:55.118494 7852 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:55.118588 7852 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 49b677342267486e8fe59d12d1905f28: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:55.118834 7854 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:55.118932 7854 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P d1263692f7d44745a587333cd534fdd7: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:37:55.119328 7848 tablet_replica.cc:439] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } } }
I20260620 11:37:55.119422 7848 tablet_replica.cc:442] T bebc5bdc5ecb47b580e8ca7c6fa03b72 P 055f36cc7ef24940b3515ae00577f32b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:37:55.119463 7871 mvcc.cc:205] Tried to move back new op lower bound from 7298889626062553088 to 7298889625802440704. Current Snapshot: MvccSnapshot[applied={T|T < 7298889626062553088}]
I20260620 11:37:55.119583 7870 mvcc.cc:205] Tried to move back new op lower bound from 7298889626062553088 to 7298889625802440704. Current Snapshot: MvccSnapshot[applied={T|T < 7298889626062553088 or (T in {7298889626062553088})}]
I20260620 11:37:55.119678 7872 mvcc.cc:205] Tried to move back new op lower bound from 7298889626062553088 to 7298889625802440704. Current Snapshot: MvccSnapshot[applied={T|T < 7298889626062553088}]
I20260620 11:37:55.125191 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.122397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.5.122.131:41235)
0620 11:37:55.122467 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.122470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.122590 (+ 120us) server_negotiation.cc:407] Connection header received
0620 11:37:55.122660 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.122660 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.122669 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.122712 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:55.122857 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.123329 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.124506 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.124695 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.124756 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.124936 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.124966 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.125009 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.125084 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.125102 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:37:55.130827 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.128046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:57049 (local address 127.5.122.131:41235)
0620 11:37:55.128091 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.128093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.128216 (+ 123us) server_negotiation.cc:407] Connection header received
0620 11:37:55.128279 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.128280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.128288 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.128358 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:55.128513 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.129027 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.130159 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.130336 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.130391 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.130613 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.130640 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.130677 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.130736 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.130752 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:37:55.130985 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.127999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:41235 (local address 127.5.122.131:57049)
0620 11:37:55.128142 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:55.128154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:55.128218 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:55.128364 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:55.128365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:55.128367 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:37:55.128436 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.128438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.129109 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.129110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.130081 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.130084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.130414 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.130415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.130485 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.130503 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.130682 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.130683 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 11:37:55.130735 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:37:55.130784 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260620 11:37:55.133426 7495 raft_consensus.cc:1261] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:55.133433 7635 raft_consensus.cc:1261] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:55.133679 7883 consensus_queue.cc:1035] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:55.133790 7851 consensus_queue.cc:1035] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:55.146617 7337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:55.146720 7337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:55.147502 7337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Sending DeleteTablet for 3 replicas of tablet 793aa7b13f2d44689750dd1665e5ee4f
I20260620 11:37:55.147786 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet 793aa7b13f2d44689750dd1665e5ee4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:55 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:59778
I20260620 11:37:55.147819 7615 tablet_service.cc:1514] Processing DeleteTablet for tablet 793aa7b13f2d44689750dd1665e5ee4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:55 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:49734
I20260620 11:37:55.148135 7887 tablet_replica.cc:330] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: stopping tablet replica
I20260620 11:37:55.148118 7475 tablet_service.cc:1514] Processing DeleteTablet for tablet 793aa7b13f2d44689750dd1665e5ee4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:55 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58620
I20260620 11:37:55.148321 7888 tablet_replica.cc:330] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: stopping tablet replica
I20260620 11:37:55.148339 7889 tablet_replica.cc:330] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: stopping tablet replica
I20260620 11:37:55.148450 7889 raft_consensus.cc:2229] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:55.148393 7337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 11:37:55.148530 7889 raft_consensus.cc:2258] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 11:37:55.148533 7337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:55.148756 7888 raft_consensus.cc:2229] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:55.148922 7887 raft_consensus.cc:2229] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:37:55.148974 7889 ts_tablet_manager.cc:1905] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:55.149138 7887 raft_consensus.cc:2258] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:55.149250 7888 raft_consensus.cc:2258] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:55.149927 7475 tablet_service.cc:1467] Processing CreateTablet for tablet 5948c216894a41f6bbc0646a182ee526 (DEFAULT_TABLE table=test-table [id=fdc8b162b447417da6a089893b315ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:55.150084 7475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5948c216894a41f6bbc0646a182ee526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:55.150189 7887 ts_tablet_manager.cc:1905] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:55.151353 7888 ts_tablet_manager.cc:1905] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:55.151609 7615 tablet_service.cc:1467] Processing CreateTablet for tablet 5948c216894a41f6bbc0646a182ee526 (DEFAULT_TABLE table=test-table [id=fdc8b162b447417da6a089893b315ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:55.151737 7615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5948c216894a41f6bbc0646a182ee526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:55.152307 7887 ts_tablet_manager.cc:1918] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:55.152433 7887 log.cc:1196] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/793aa7b13f2d44689750dd1665e5ee4f
I20260620 11:37:55.152782 7887 ts_tablet_manager.cc:1939] T 793aa7b13f2d44689750dd1665e5ee4f P d1263692f7d44745a587333cd534fdd7: Deleting consensus metadata
I20260620 11:37:55.153321 7845 tablet_bootstrap.cc:492] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: Bootstrap starting.
I20260620 11:37:55.153355 7849 tablet_bootstrap.cc:492] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: Bootstrap starting.
I20260620 11:37:55.153564 7888 ts_tablet_manager.cc:1918] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:55.153630 7888 log.cc:1196] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/793aa7b13f2d44689750dd1665e5ee4f
I20260620 11:37:55.153766 7845 tablet_bootstrap.cc:654] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:55.153766 7849 tablet_bootstrap.cc:654] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:55.153894 7888 ts_tablet_manager.cc:1939] T 793aa7b13f2d44689750dd1665e5ee4f P 055f36cc7ef24940b3515ae00577f32b: Deleting consensus metadata
I20260620 11:37:55.154074 7889 ts_tablet_manager.cc:1918] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:55.154140 7889 log.cc:1196] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/793aa7b13f2d44689750dd1665e5ee4f
I20260620 11:37:55.154378 7889 ts_tablet_manager.cc:1939] T 793aa7b13f2d44689750dd1665e5ee4f P 49b677342267486e8fe59d12d1905f28: Deleting consensus metadata
I20260620 11:37:55.154459 7324 catalog_manager.cc:4841] TS d1263692f7d44745a587333cd534fdd7 (127.5.122.131:41235): tablet 793aa7b13f2d44689750dd1665e5ee4f (table test-table [id=7003db0134b74a31892200998df9c1ae]) successfully deleted
I20260620 11:37:55.154654 7845 tablet_bootstrap.cc:492] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: No bootstrap required, opened a new log
I20260620 11:37:55.154672 7755 tablet_service.cc:1467] Processing CreateTablet for tablet 5948c216894a41f6bbc0646a182ee526 (DEFAULT_TABLE table=test-table [id=fdc8b162b447417da6a089893b315ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:55.154695 7845 ts_tablet_manager.cc:1397] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 11:37:55.154772 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5948c216894a41f6bbc0646a182ee526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:55.154812 7322 catalog_manager.cc:4841] TS 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981): tablet 793aa7b13f2d44689750dd1665e5ee4f (table test-table [id=7003db0134b74a31892200998df9c1ae]) successfully deleted
I20260620 11:37:55.154861 7845 raft_consensus.cc:348] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.154942 7845 raft_consensus.cc:374] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:55.154961 7845 raft_consensus.cc:729] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055f36cc7ef24940b3515ae00577f32b, State: Initialized, Role: FOLLOWER
I20260620 11:37:55.155009 7845 consensus_queue.cc:260] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.155088 7845 ts_tablet_manager.cc:1428] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 11:37:55.155316 7322 catalog_manager.cc:4841] TS 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065): tablet 793aa7b13f2d44689750dd1665e5ee4f (table test-table [id=7003db0134b74a31892200998df9c1ae]) successfully deleted
I20260620 11:37:55.156206 7844 tablet_bootstrap.cc:492] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: Bootstrap starting.
I20260620 11:37:55.156643 7844 tablet_bootstrap.cc:654] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:55.156999 7849 tablet_bootstrap.cc:492] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: No bootstrap required, opened a new log
I20260620 11:37:55.157063 7849 ts_tablet_manager.cc:1397] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260620 11:37:55.157254 7844 tablet_bootstrap.cc:492] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: No bootstrap required, opened a new log
I20260620 11:37:55.157233 7849 raft_consensus.cc:348] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.157306 7849 raft_consensus.cc:374] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:55.157303 7844 ts_tablet_manager.cc:1397] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:55.157335 7849 raft_consensus.cc:729] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b677342267486e8fe59d12d1905f28, State: Initialized, Role: FOLLOWER
I20260620 11:37:55.157385 7849 consensus_queue.cc:260] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.157477 7849 ts_tablet_manager.cc:1428] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 11:37:55.157465 7844 raft_consensus.cc:348] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.157552 7844 raft_consensus.cc:374] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:55.157579 7844 raft_consensus.cc:729] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Initialized, Role: FOLLOWER
I20260620 11:37:55.157621 7844 consensus_queue.cc:260] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.157724 7844 ts_tablet_manager.cc:1428] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:55.197271 7852 raft_consensus.cc:482] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:55.197398 7852 raft_consensus.cc:504] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.197773 7852 leader_election.cc:288] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065), d1263692f7d44745a587333cd534fdd7 (127.5.122.131:41235)
I20260620 11:37:55.201323 7893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.197994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:45065 (local address 127.5.122.129:59527)
0620 11:37:55.198306 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:55.198317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:55.198400 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:55.198533 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:55.198533 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:55.198535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:55.198614 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.198615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.199605 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.199611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.200440 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.200442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.201045 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.201046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.201120 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.201125 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:37:55.201186 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 11:37:55.201229 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":18,"thread_start_us":239,"threads_started":1}
I20260620 11:37:55.201321 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.198103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:59527 (local address 127.5.122.130:45065)
0620 11:37:55.198152 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.198154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.198385 (+ 231us) server_negotiation.cc:407] Connection header received
0620 11:37:55.198459 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.198460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.198471 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.198528 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:55.198683 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.199464 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.200547 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.200954 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.201039 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.201048 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.201204 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.201249 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:37:55.201566 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5948c216894a41f6bbc0646a182ee526" candidate_uuid: "49b677342267486e8fe59d12d1905f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b" is_pre_election: true
I20260620 11:37:55.201665 7635 raft_consensus.cc:2454] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b677342267486e8fe59d12d1905f28 in term 0.
I20260620 11:37:55.201854 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.198133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:33719 (local address 127.5.122.131:41235)
0620 11:37:55.198184 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.198187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.198396 (+ 209us) server_negotiation.cc:407] Connection header received
0620 11:37:55.198512 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.198513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.198522 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.198554 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:37:55.198769 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.199464 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.200930 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.201509 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.201580 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.201589 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.201740 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.201793 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:37:55.201872 7426 leader_election.cc:302] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055f36cc7ef24940b3515ae00577f32b, 49b677342267486e8fe59d12d1905f28; no voters:
I20260620 11:37:55.202081 7852 raft_consensus.cc:2781] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:55.202121 7852 raft_consensus.cc:482] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:55.202152 7852 raft_consensus.cc:3037] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:55.202255 7894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.198011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:41235 (local address 127.5.122.129:33719)
0620 11:37:55.198320 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:55.198331 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:55.198425 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:55.198622 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:55.198623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:55.198624 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:37:55.198699 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.198700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.199561 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.199562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.200832 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.200836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.201580 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.201582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.201653 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.201657 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:37:55.202058 (+ 401us) client_negotiation.cc:240] Negotiation successful
0620 11:37:55.202110 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":52,"thread_start_us":239,"threads_started":1}
I20260620 11:37:55.202935 7852 raft_consensus.cc:504] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.203102 7852 leader_election.cc:288] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [CANDIDATE]: Term 1 election: Requested vote from peers 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065), d1263692f7d44745a587333cd534fdd7 (127.5.122.131:41235)
I20260620 11:37:55.203202 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5948c216894a41f6bbc0646a182ee526" candidate_uuid: "49b677342267486e8fe59d12d1905f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1263692f7d44745a587333cd534fdd7" is_pre_election: true
I20260620 11:37:55.203342 7775 raft_consensus.cc:2454] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b677342267486e8fe59d12d1905f28 in term 0.
I20260620 11:37:55.203353 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5948c216894a41f6bbc0646a182ee526" candidate_uuid: "49b677342267486e8fe59d12d1905f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b"
I20260620 11:37:55.203428 7635 raft_consensus.cc:3037] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:55.204241 7635 raft_consensus.cc:2454] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49b677342267486e8fe59d12d1905f28 in term 1.
I20260620 11:37:55.204432 7426 leader_election.cc:302] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055f36cc7ef24940b3515ae00577f32b, 49b677342267486e8fe59d12d1905f28; no voters:
I20260620 11:37:55.204555 7852 raft_consensus.cc:2781] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:55.204593 7775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5948c216894a41f6bbc0646a182ee526" candidate_uuid: "49b677342267486e8fe59d12d1905f28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1263692f7d44745a587333cd534fdd7"
I20260620 11:37:55.204691 7775 raft_consensus.cc:3037] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:55.204728 7852 raft_consensus.cc:686] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 1 LEADER]: Becoming Leader. State: Replica: 49b677342267486e8fe59d12d1905f28, State: Running, Role: LEADER
I20260620 11:37:55.204810 7852 consensus_queue.cc:237] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:55.205467 7775 raft_consensus.cc:2454] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49b677342267486e8fe59d12d1905f28 in term 1.
I20260620 11:37:55.205643 7337 catalog_manager.cc:5495] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49b677342267486e8fe59d12d1905f28 (127.5.122.129). New cstate: current_term: 1 leader_uuid: "49b677342267486e8fe59d12d1905f28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } health_report { overall_health: UNKNOWN } } }
I20260620 11:37:55.221617 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.219220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.5.122.129:32981)
0620 11:37:55.219267 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.219269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.219278 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:37:55.219321 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.219321 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.219344 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.219429 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:55.219553 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.220041 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.220779 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.220997 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.221080 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.221251 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.221375 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.221460 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.221501 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.221524 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:37:55.227296 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.224288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:49923 (local address 127.5.122.131:41235)
0620 11:37:55.224334 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.224336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.224440 (+ 104us) server_negotiation.cc:407] Connection header received
0620 11:37:55.224525 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.224526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.224536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.224576 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:55.224727 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.225179 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.226306 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.226497 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.226598 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.227065 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.227095 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.227135 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.227214 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.227235 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:37:55.227461 7899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.224197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:41235 (local address 127.5.122.129:49923)
0620 11:37:55.224381 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:55.224394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:55.224457 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:55.224594 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:55.224595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:55.224596 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:37:55.224673 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.224674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.225253 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.225254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.226227 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.226230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.226877 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.226878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.226947 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.226988 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.227154 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.227156 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:37:55.227198 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 11:37:55.227248 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260620 11:37:55.232451 7635 raft_consensus.cc:1261] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Refusing update from remote peer 49b677342267486e8fe59d12d1905f28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:55.232586 7775 raft_consensus.cc:1261] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Refusing update from remote peer 49b677342267486e8fe59d12d1905f28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:55.232721 7852 consensus_queue.cc:1035] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:55.233016 7852 consensus_queue.cc:1035] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:55.240324 7882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.237465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:32981 (local address 127.5.122.131:57805)
0620 11:37:55.237516 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:55.237528 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:37:55.237567 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:37:55.237837 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:37:55.237838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:37:55.237840 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:37:55.237887 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.237888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.238836 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.238837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.239519 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:37:55.239521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.239884 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.239885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:37:55.239948 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.239954 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.240143 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.240145 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:37:55.240210 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 11:37:55.240255 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 11:37:55.240320 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:55.237658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:57805 (local address 127.5.122.129:32981)
0620 11:37:55.237723 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:55.237726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:55.237734 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:37:55.237755 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:55.237755 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:55.237766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:55.237823 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:55.238001 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.238748 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.239614 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:55.239827 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:55.239889 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:55.240040 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:55.240075 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:55.240121 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:55.240214 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 11:37:55.240245 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260620 11:37:55.240689 7871 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 11:37:56.763294 7337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:56.763423 7337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:56.764213 7337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Sending DeleteTablet for 3 replicas of tablet 5948c216894a41f6bbc0646a182ee526
I20260620 11:37:56.764600 7475 tablet_service.cc:1514] Processing DeleteTablet for tablet 5948c216894a41f6bbc0646a182ee526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:56 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58620
I20260620 11:37:56.764603 7615 tablet_service.cc:1514] Processing DeleteTablet for tablet 5948c216894a41f6bbc0646a182ee526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:56 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:49734
I20260620 11:37:56.764811 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet 5948c216894a41f6bbc0646a182ee526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:56 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:59778
I20260620 11:37:56.766086 7924 tablet_replica.cc:330] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: stopping tablet replica
I20260620 11:37:56.766172 7924 raft_consensus.cc:2229] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:37:56.766301 7924 raft_consensus.cc:2258] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:56.766911 7924 ts_tablet_manager.cc:1905] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:56.768205 7926 tablet_replica.cc:330] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: stopping tablet replica
I20260620 11:37:56.768306 7926 raft_consensus.cc:2229] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:56.768360 7926 raft_consensus.cc:2258] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:56.768494 7925 tablet_replica.cc:330] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: stopping tablet replica
I20260620 11:37:56.768543 7924 ts_tablet_manager.cc:1918] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 11:37:56.768568 7925 raft_consensus.cc:2229] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:56.768610 7925 raft_consensus.cc:2258] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:56.768673 7924 log.cc:1196] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/5948c216894a41f6bbc0646a182ee526
I20260620 11:37:56.768926 7925 ts_tablet_manager.cc:1905] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:56.768991 7924 ts_tablet_manager.cc:1939] T 5948c216894a41f6bbc0646a182ee526 P 49b677342267486e8fe59d12d1905f28: Deleting consensus metadata
I20260620 11:37:56.769434 7322 catalog_manager.cc:4841] TS 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981): tablet 5948c216894a41f6bbc0646a182ee526 (table test-table [id=fdc8b162b447417da6a089893b315ed1]) successfully deleted
I20260620 11:37:56.770160 7926 ts_tablet_manager.cc:1905] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:56.770849 7925 ts_tablet_manager.cc:1918] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 11:37:56.770921 7925 log.cc:1196] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/5948c216894a41f6bbc0646a182ee526
I20260620 11:37:56.771178 7925 ts_tablet_manager.cc:1939] T 5948c216894a41f6bbc0646a182ee526 P 055f36cc7ef24940b3515ae00577f32b: Deleting consensus metadata
I20260620 11:37:56.771533 7322 catalog_manager.cc:4841] TS 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065): tablet 5948c216894a41f6bbc0646a182ee526 (table test-table [id=fdc8b162b447417da6a089893b315ed1]) successfully deleted
I20260620 11:37:56.771643 7926 ts_tablet_manager.cc:1918] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 11:37:56.771744 7926 log.cc:1196] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/5948c216894a41f6bbc0646a182ee526
I20260620 11:37:56.771914 7926 ts_tablet_manager.cc:1939] T 5948c216894a41f6bbc0646a182ee526 P d1263692f7d44745a587333cd534fdd7: Deleting consensus metadata
I20260620 11:37:56.772679 7324 catalog_manager.cc:4841] TS d1263692f7d44745a587333cd534fdd7 (127.5.122.131:41235): tablet 5948c216894a41f6bbc0646a182ee526 (table test-table [id=fdc8b162b447417da6a089893b315ed1]) successfully deleted
Jun 20 11:37:56 dist-test-slave-szdr krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 11:37:56 dist-test-slave-szdr krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:56.790098 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:56.782931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.5.122.190:37155)
0620 11:37:56.783080 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:56.783082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:56.783135 (+ 53us) server_negotiation.cc:407] Connection header received
0620 11:37:56.783188 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:56.783188 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:56.783200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:56.783269 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:56.783907 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:56.784485 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:56.785207 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:56.785410 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:56.785461 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:56.787795 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:56.787797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:56.787798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:56.787806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:56.789092 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:56.789403 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:56.789404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:56.789404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:56.789458 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:56.789612 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:56.789612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:56.789613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:56.789721 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:56.789797 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:56.789882 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 11:37:56.789947 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260620 11:37:56.791723 7337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:37:56.791888 7337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:56.793306 7475 tablet_service.cc:1467] Processing CreateTablet for tablet 2103a85af7a3414e9986adf4e85ea34c (DEFAULT_TABLE table=test-table [id=2ad62a8e9f2b464aa38a38251f1224ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:56.793344 7755 tablet_service.cc:1467] Processing CreateTablet for tablet 2103a85af7a3414e9986adf4e85ea34c (DEFAULT_TABLE table=test-table [id=2ad62a8e9f2b464aa38a38251f1224ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:56.793460 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2103a85af7a3414e9986adf4e85ea34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:56.793462 7475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2103a85af7a3414e9986adf4e85ea34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:56.794114 7615 tablet_service.cc:1467] Processing CreateTablet for tablet 2103a85af7a3414e9986adf4e85ea34c (DEFAULT_TABLE table=test-table [id=2ad62a8e9f2b464aa38a38251f1224ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:56.794281 7615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2103a85af7a3414e9986adf4e85ea34c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:56.794975 7936 tablet_bootstrap.cc:492] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: Bootstrap starting.
I20260620 11:37:56.795373 7936 tablet_bootstrap.cc:654] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:56.795562 7937 tablet_bootstrap.cc:492] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: Bootstrap starting.
I20260620 11:37:56.795732 7935 tablet_bootstrap.cc:492] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: Bootstrap starting.
I20260620 11:37:56.796016 7937 tablet_bootstrap.cc:654] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:56.796085 7935 tablet_bootstrap.cc:654] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:56.796103 7936 tablet_bootstrap.cc:492] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: No bootstrap required, opened a new log
I20260620 11:37:56.796151 7936 ts_tablet_manager.cc:1397] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:56.796281 7936 raft_consensus.cc:348] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.796383 7936 raft_consensus.cc:374] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:56.796437 7936 raft_consensus.cc:729] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b677342267486e8fe59d12d1905f28, State: Initialized, Role: FOLLOWER
I20260620 11:37:56.796512 7936 consensus_queue.cc:260] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.796646 7936 ts_tablet_manager.cc:1428] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:56.796644 7937 tablet_bootstrap.cc:492] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: No bootstrap required, opened a new log
I20260620 11:37:56.796696 7937 ts_tablet_manager.cc:1397] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:56.796868 7937 raft_consensus.cc:348] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.796941 7937 raft_consensus.cc:374] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:56.796962 7937 raft_consensus.cc:729] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055f36cc7ef24940b3515ae00577f32b, State: Initialized, Role: FOLLOWER
I20260620 11:37:56.797004 7937 consensus_queue.cc:260] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.797188 7937 ts_tablet_manager.cc:1428] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:56.797853 7935 tablet_bootstrap.cc:492] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: No bootstrap required, opened a new log
I20260620 11:37:56.798002 7935 ts_tablet_manager.cc:1397] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:56.798143 7935 raft_consensus.cc:348] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.798213 7935 raft_consensus.cc:374] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:56.798226 7935 raft_consensus.cc:729] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Initialized, Role: FOLLOWER
I20260620 11:37:56.798287 7935 consensus_queue.cc:260] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.798390 7935 ts_tablet_manager.cc:1428] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:56.812563 7883 raft_consensus.cc:482] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:56.812680 7883 raft_consensus.cc:504] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.812844 7883 leader_election.cc:288] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981), 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065)
I20260620 11:37:56.813094 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2103a85af7a3414e9986adf4e85ea34c" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28" is_pre_election: true
I20260620 11:37:56.813186 7495 raft_consensus.cc:2454] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:56.813309 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2103a85af7a3414e9986adf4e85ea34c" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b" is_pre_election: true
I20260620 11:37:56.813362 7706 leader_election.cc:302] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b677342267486e8fe59d12d1905f28, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:56.813412 7635 raft_consensus.cc:2454] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:56.813465 7883 raft_consensus.cc:2781] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:56.813503 7883 raft_consensus.cc:482] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:56.813524 7883 raft_consensus.cc:3037] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:56.814400 7883 raft_consensus.cc:504] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.814546 7883 leader_election.cc:288] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Requested vote from peers 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981), 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065)
I20260620 11:37:56.814752 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2103a85af7a3414e9986adf4e85ea34c" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28"
I20260620 11:37:56.814831 7495 raft_consensus.cc:3037] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:56.814756 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2103a85af7a3414e9986adf4e85ea34c" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b"
I20260620 11:37:56.814881 7635 raft_consensus.cc:3037] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:56.815529 7495 raft_consensus.cc:2454] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:56.815562 7635 raft_consensus.cc:2454] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:56.815729 7706 leader_election.cc:302] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055f36cc7ef24940b3515ae00577f32b, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:56.815919 7883 raft_consensus.cc:2781] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:56.815982 7883 raft_consensus.cc:686] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 1 LEADER]: Becoming Leader. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Running, Role: LEADER
I20260620 11:37:56.816058 7883 consensus_queue.cc:237] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.816694 7338 catalog_manager.cc:5495] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1263692f7d44745a587333cd534fdd7 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } health_report { overall_health: HEALTHY } } }
I20260620 11:37:56.829077 7944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:56.826622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37800 (local address 127.5.122.131:41235)
0620 11:37:56.826781 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:56.826783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:56.826790 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:37:56.826818 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:56.826818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:56.826828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:56.826919 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:37:56.827047 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:56.827549 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:56.828335 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:56.828559 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:56.828631 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:56.828771 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:37:56.828813 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:37:56.828891 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:56.828954 (+ 63us) server_negotiation.cc:299] Negotiation successful
0620 11:37:56.828976 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260620 11:37:56.831523 7495 raft_consensus.cc:1261] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:56.831523 7635 raft_consensus.cc:1261] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:56.831780 7906 consensus_queue.cc:1035] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:56.831892 7883 consensus_queue.cc:1035] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:56.841770 7337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:56.841873 7337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:56.842532 7337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Sending DeleteTablet for 3 replicas of tablet 2103a85af7a3414e9986adf4e85ea34c
I20260620 11:37:56.842801 7615 tablet_service.cc:1514] Processing DeleteTablet for tablet 2103a85af7a3414e9986adf4e85ea34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:56 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:49734
I20260620 11:37:56.842859 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet 2103a85af7a3414e9986adf4e85ea34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:56 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:59778
I20260620 11:37:56.842911 7925 tablet_replica.cc:330] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: stopping tablet replica
I20260620 11:37:56.842934 7475 tablet_service.cc:1514] Processing DeleteTablet for tablet 2103a85af7a3414e9986adf4e85ea34c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:56 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58620
I20260620 11:37:56.842963 7926 tablet_replica.cc:330] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: stopping tablet replica
I20260620 11:37:56.842958 7925 raft_consensus.cc:2229] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:56.843010 7925 raft_consensus.cc:2258] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:56.843019 7926 raft_consensus.cc:2229] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:37:56.843032 7924 tablet_replica.cc:330] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: stopping tablet replica
I20260620 11:37:56.843086 7926 raft_consensus.cc:2258] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:56.843101 7924 raft_consensus.cc:2229] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:56.843142 7924 raft_consensus.cc:2258] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:56.843145 7337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:37:56.843286 7337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:37:56.843367 7925 ts_tablet_manager.cc:1905] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:56.843467 7924 ts_tablet_manager.cc:1905] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:56.844447 7926 ts_tablet_manager.cc:1905] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:56.844522 7755 tablet_service.cc:1467] Processing CreateTablet for tablet ea2873eb3a98471aab07c01683a45da5 (DEFAULT_TABLE table=test-table [id=1de52695e434450f87ba98cd6978f97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:56.844624 7755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea2873eb3a98471aab07c01683a45da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:56.845175 7925 ts_tablet_manager.cc:1918] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:56.845218 7925 log.cc:1196] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/2103a85af7a3414e9986adf4e85ea34c
I20260620 11:37:56.845403 7925 ts_tablet_manager.cc:1939] T 2103a85af7a3414e9986adf4e85ea34c P 055f36cc7ef24940b3515ae00577f32b: Deleting consensus metadata
I20260620 11:37:56.845674 7924 ts_tablet_manager.cc:1918] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:56.845718 7924 log.cc:1196] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/2103a85af7a3414e9986adf4e85ea34c
I20260620 11:37:56.845935 7924 ts_tablet_manager.cc:1939] T 2103a85af7a3414e9986adf4e85ea34c P 49b677342267486e8fe59d12d1905f28: Deleting consensus metadata
I20260620 11:37:56.846019 7935 tablet_bootstrap.cc:492] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: Bootstrap starting.
I20260620 11:37:56.846176 7926 ts_tablet_manager.cc:1918] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:37:56.846235 7926 log.cc:1196] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/2103a85af7a3414e9986adf4e85ea34c
I20260620 11:37:56.846266 7322 catalog_manager.cc:4841] TS 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065): tablet 2103a85af7a3414e9986adf4e85ea34c (table test-table [id=2ad62a8e9f2b464aa38a38251f1224ee]) successfully deleted
I20260620 11:37:56.846395 7935 tablet_bootstrap.cc:654] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:56.846462 7926 ts_tablet_manager.cc:1939] T 2103a85af7a3414e9986adf4e85ea34c P d1263692f7d44745a587333cd534fdd7: Deleting consensus metadata
I20260620 11:37:56.846489 7475 tablet_service.cc:1467] Processing CreateTablet for tablet ea2873eb3a98471aab07c01683a45da5 (DEFAULT_TABLE table=test-table [id=1de52695e434450f87ba98cd6978f97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:56.846581 7475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea2873eb3a98471aab07c01683a45da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:56.846721 7615 tablet_service.cc:1467] Processing CreateTablet for tablet ea2873eb3a98471aab07c01683a45da5 (DEFAULT_TABLE table=test-table [id=1de52695e434450f87ba98cd6978f97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:37:56.846761 7324 catalog_manager.cc:4841] TS d1263692f7d44745a587333cd534fdd7 (127.5.122.131:41235): tablet 2103a85af7a3414e9986adf4e85ea34c (table test-table [id=2ad62a8e9f2b464aa38a38251f1224ee]) successfully deleted
I20260620 11:37:56.846830 7615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea2873eb3a98471aab07c01683a45da5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:56.847657 7936 tablet_bootstrap.cc:492] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28: Bootstrap starting.
I20260620 11:37:56.847664 7937 tablet_bootstrap.cc:492] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: Bootstrap starting.
I20260620 11:37:56.847890 7322 catalog_manager.cc:4841] TS 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981): tablet 2103a85af7a3414e9986adf4e85ea34c (table test-table [id=2ad62a8e9f2b464aa38a38251f1224ee]) successfully deleted
I20260620 11:37:56.847924 7935 tablet_bootstrap.cc:492] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: No bootstrap required, opened a new log
I20260620 11:37:56.847978 7935 ts_tablet_manager.cc:1397] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:37:56.848079 7936 tablet_bootstrap.cc:654] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:56.848134 7935 raft_consensus.cc:348] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.848213 7935 raft_consensus.cc:374] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:56.848232 7935 raft_consensus.cc:729] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Initialized, Role: FOLLOWER
I20260620 11:37:56.848284 7935 consensus_queue.cc:260] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.848341 7937 tablet_bootstrap.cc:654] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:56.848412 7935 ts_tablet_manager.cc:1428] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:56.848726 7936 tablet_bootstrap.cc:492] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28: No bootstrap required, opened a new log
I20260620 11:37:56.848793 7936 ts_tablet_manager.cc:1397] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:56.848978 7937 tablet_bootstrap.cc:492] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: No bootstrap required, opened a new log
I20260620 11:37:56.849040 7937 ts_tablet_manager.cc:1397] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:37:56.849061 7936 raft_consensus.cc:348] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.849134 7936 raft_consensus.cc:374] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:56.849155 7936 raft_consensus.cc:729] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b677342267486e8fe59d12d1905f28, State: Initialized, Role: FOLLOWER
I20260620 11:37:56.849205 7936 consensus_queue.cc:260] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.849187 7937 raft_consensus.cc:348] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.849272 7937 raft_consensus.cc:374] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:56.849298 7937 raft_consensus.cc:729] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055f36cc7ef24940b3515ae00577f32b, State: Initialized, Role: FOLLOWER
I20260620 11:37:56.849319 7936 ts_tablet_manager.cc:1428] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:56.849345 7937 consensus_queue.cc:260] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.849443 7937 ts_tablet_manager.cc:1428] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:56.984114 7911 raft_consensus.cc:482] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:37:56.984221 7911 raft_consensus.cc:504] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.984421 7911 leader_election.cc:288] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981), 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065)
I20260620 11:37:56.984678 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2873eb3a98471aab07c01683a45da5" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28" is_pre_election: true
I20260620 11:37:56.984786 7495 raft_consensus.cc:2454] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:56.984711 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2873eb3a98471aab07c01683a45da5" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b" is_pre_election: true
I20260620 11:37:56.984872 7635 raft_consensus.cc:2454] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 0.
I20260620 11:37:56.985006 7706 leader_election.cc:302] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b677342267486e8fe59d12d1905f28, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:56.985126 7911 raft_consensus.cc:2781] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:37:56.985195 7911 raft_consensus.cc:482] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:37:56.985227 7911 raft_consensus.cc:3037] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:56.986073 7911 raft_consensus.cc:504] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.986217 7911 leader_election.cc:288] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Requested vote from peers 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981), 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065)
I20260620 11:37:56.986408 7635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2873eb3a98471aab07c01683a45da5" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055f36cc7ef24940b3515ae00577f32b"
I20260620 11:37:56.986410 7495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea2873eb3a98471aab07c01683a45da5" candidate_uuid: "d1263692f7d44745a587333cd534fdd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49b677342267486e8fe59d12d1905f28"
I20260620 11:37:56.986487 7635 raft_consensus.cc:3037] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:56.986502 7495 raft_consensus.cc:3037] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:56.987112 7495 raft_consensus.cc:2454] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:56.987143 7635 raft_consensus.cc:2454] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1263692f7d44745a587333cd534fdd7 in term 1.
I20260620 11:37:56.987305 7706 leader_election.cc:302] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055f36cc7ef24940b3515ae00577f32b, d1263692f7d44745a587333cd534fdd7; no voters:
I20260620 11:37:56.987433 7911 raft_consensus.cc:2781] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:56.987516 7911 raft_consensus.cc:686] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 1 LEADER]: Becoming Leader. State: Replica: d1263692f7d44745a587333cd534fdd7, State: Running, Role: LEADER
I20260620 11:37:56.987576 7911 consensus_queue.cc:237] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } }
I20260620 11:37:56.988186 7337 catalog_manager.cc:5495] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1263692f7d44745a587333cd534fdd7 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "d1263692f7d44745a587333cd534fdd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1263692f7d44745a587333cd534fdd7" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41235 } health_report { overall_health: HEALTHY } } }
I20260620 11:37:56.999888 7495 raft_consensus.cc:1261] T ea2873eb3a98471aab07c01683a45da5 P 49b677342267486e8fe59d12d1905f28 [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:56.999888 7635 raft_consensus.cc:1261] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Refusing update from remote peer d1263692f7d44745a587333cd534fdd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:37:57.000140 7911 consensus_queue.cc:1035] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "055f36cc7ef24940b3515ae00577f32b" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:37:57.000253 7883 consensus_queue.cc:1035] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49b677342267486e8fe59d12d1905f28" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 11:37:57.004611 7904 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 11:37:58.522217 7337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:58.522323 7337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:37:58.523110 7337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a46257f10b9f46f18ca2bc1da2956801: Sending DeleteTablet for 3 replicas of tablet ea2873eb3a98471aab07c01683a45da5
I20260620 11:37:58.523396 7615 tablet_service.cc:1514] Processing DeleteTablet for tablet ea2873eb3a98471aab07c01683a45da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:58 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:49734
I20260620 11:37:58.523440 7755 tablet_service.cc:1514] Processing DeleteTablet for tablet ea2873eb3a98471aab07c01683a45da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:58 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:59778
I20260620 11:37:58.523512 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 7410
I20260620 11:37:58.523631 7475 tablet_service.cc:1514] Processing DeleteTablet for tablet ea2873eb3a98471aab07c01683a45da5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:37:58 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58620
I20260620 11:37:58.524085 7974 tablet_replica.cc:330] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: stopping tablet replica
I20260620 11:37:58.524149 7974 raft_consensus.cc:2229] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:37:58.524192 7974 raft_consensus.cc:2258] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:58.524451 7974 ts_tablet_manager.cc:1905] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:58.524736 7975 tablet_replica.cc:330] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: stopping tablet replica
I20260620 11:37:58.524843 7975 raft_consensus.cc:2229] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:37:58.524955 7975 raft_consensus.cc:2258] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:37:58.525272 7975 ts_tablet_manager.cc:1905] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:37:58.526466 7974 ts_tablet_manager.cc:1918] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 11:37:58.526561 7974 log.cc:1196] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/ea2873eb3a98471aab07c01683a45da5
I20260620 11:37:58.526854 7974 ts_tablet_manager.cc:1939] T ea2873eb3a98471aab07c01683a45da5 P 055f36cc7ef24940b3515ae00577f32b: Deleting consensus metadata
I20260620 11:37:58.527247 7322 catalog_manager.cc:4841] TS 055f36cc7ef24940b3515ae00577f32b (127.5.122.130:45065): tablet ea2873eb3a98471aab07c01683a45da5 (table test-table [id=1de52695e434450f87ba98cd6978f97b]) successfully deleted
I20260620 11:37:58.527371 7975 ts_tablet_manager.cc:1918] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 11:37:58.527439 7975 log.cc:1196] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/ea2873eb3a98471aab07c01683a45da5
I20260620 11:37:58.527696 7975 ts_tablet_manager.cc:1939] T ea2873eb3a98471aab07c01683a45da5 P d1263692f7d44745a587333cd534fdd7: Deleting consensus metadata
I20260620 11:37:58.528013 7324 catalog_manager.cc:4841] TS d1263692f7d44745a587333cd534fdd7 (127.5.122.131:41235): tablet ea2873eb3a98471aab07c01683a45da5 (table test-table [id=1de52695e434450f87ba98cd6978f97b]) successfully deleted
W20260620 11:37:58.530752 7322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:32981 (error 108)
I20260620 11:37:58.531008 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 7550
I20260620 11:37:58.531798 7977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:58.531141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:32981 (local address 127.0.0.1:38540)
0620 11:37:58.531701 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0620 11:37:58.531755 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:32981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":58,"threads_started":1}
W20260620 11:37:58.531930 7322 catalog_manager.cc:4584] TS 49b677342267486e8fe59d12d1905f28 (127.5.122.129:32981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea2873eb3a98471aab07c01683a45da5: Network error: Client connection negotiation failed: client connection to 127.5.122.129:32981: connect: Connection refused (error 111)
I20260620 11:37:58.536262 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 7690
I20260620 11:37:58.542220 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 7306
2026-06-20T11:37:58Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6037 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:37:58 dist-test-slave-szdr krb5kdc[7980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:37:58 dist-test-slave-szdr krb5kdc[7980](info): set up 2 sockets
Jun 20 11:37:58 dist-test-slave-szdr krb5kdc[7980](info): commencing operation
krb5kdc: starting...
W20260620 11:37:59.581743 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:37:59 dist-test-slave-szdr krb5kdc[7980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:37:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:37:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:37:59.704589 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36739
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:36739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:59.789832 7993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:59.790155 7993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:59.790186 7993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:59.791479 7993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:37:59.791529 7993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:59.791544 7993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:59.791587 7993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:37:59.791615 7993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:37:59.793215 7993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:36739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:59.793509 7993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:59.793766 7993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:59.798833 8001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:59.798833 7998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:59.799000 7999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:59.799310 7993 server_base.cc:1034] running on GCE node
I20260620 11:37:59.799520 7993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:59.799785 7993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:59.800931 7993 hybrid_clock.cc:648] HybridClock initialized: now 1781955479800915 us; error 34 us; skew 500 ppm
Jun 20 11:37:59 dist-test-slave-szdr krb5kdc[7980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955479, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:59.803500 7993 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:37:59.803970 7993 webserver.cc:460] Webserver started at http://127.5.122.190:46533/ using document root <none> and password file <none>
I20260620 11:37:59.804216 7993 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:59.804286 7993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:59.804430 7993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:59.805518 7993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d"
format_stamp: "Formatted at 2026-06-20 11:37:59 on dist-test-slave-szdr"
server_key: "92e5015332f1e9959e164d7212621ccd"
server_key_iv: "4d789581a17e1487ada4f2be798075c7"
server_key_version: "encryptionkey@0"
I20260620 11:37:59.805886 7993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d"
format_stamp: "Formatted at 2026-06-20 11:37:59 on dist-test-slave-szdr"
server_key: "92e5015332f1e9959e164d7212621ccd"
server_key_iv: "4d789581a17e1487ada4f2be798075c7"
server_key_version: "encryptionkey@0"
I20260620 11:37:59.807745 7993 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 11:37:59.808501 8008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:59.808661 7993 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:59.808790 7993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d"
format_stamp: "Formatted at 2026-06-20 11:37:59 on dist-test-slave-szdr"
server_key: "92e5015332f1e9959e164d7212621ccd"
server_key_iv: "4d789581a17e1487ada4f2be798075c7"
server_key_version: "encryptionkey@0"
I20260620 11:37:59.808866 7993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:37:59.830111 7993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:37:59.832010 7993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:37:59.832185 7993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:37:59.836194 7993 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:36739
I20260620 11:37:59.836236 8060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:36739 every 8 connection(s)
I20260620 11:37:59.836627 7993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:37:59.837553 8061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:37:59.839648 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 7993
I20260620 11:37:59.839761 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:37:59.839922 5610 external_mini_cluster.cc:1428] Setting key b8cf2b7918dbc3bfb43c6758384836e7
I20260620 11:37:59.839960 8061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d: Bootstrap starting.
I20260620 11:37:59.840698 8061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d: Neither blocks nor log segments found. Creating new log.
I20260620 11:37:59.841018 8061 log.cc:824] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d: Log is configured to *not* fsync() on all Append() calls
I20260620 11:37:59.841747 8061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d: No bootstrap required, opened a new log
I20260620 11:37:59.843568 8061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36739 } }
I20260620 11:37:59.843739 8061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:37:59.843786 8061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45a2561cc7514bd1b0cfa3cf83f4d67d, State: Initialized, Role: FOLLOWER
I20260620 11:37:59.843938 8061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36739 } }
I20260620 11:37:59.844020 8061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:37:59.844048 8061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:37:59.844094 8061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:37:59.844858 8061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36739 } }
I20260620 11:37:59.844972 8061 leader_election.cc:302] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45a2561cc7514bd1b0cfa3cf83f4d67d; no voters:
I20260620 11:37:59.845148 8061 leader_election.cc:288] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:37:59.845207 8066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:37:59.845443 8061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:37:59.845450 8066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [term 1 LEADER]: Becoming Leader. State: Replica: 45a2561cc7514bd1b0cfa3cf83f4d67d, State: Running, Role: LEADER
I20260620 11:37:59.845854 8066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36739 } }
I20260620 11:37:59.846344 8068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45a2561cc7514bd1b0cfa3cf83f4d67d. Latest consensus state: current_term: 1 leader_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36739 } } }
I20260620 11:37:59.846454 8068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [sys.catalog]: This master's current role is: LEADER
I20260620 11:37:59.846555 8067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a2561cc7514bd1b0cfa3cf83f4d67d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36739 } } }
I20260620 11:37:59.846637 8067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d [sys.catalog]: This master's current role is: LEADER
Jun 20 11:37:59 dist-test-slave-szdr krb5kdc[7980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:37:59.847425 8075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:37:59.847949 8075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:37:59.850708 8075 catalog_manager.cc:1347] Generated new cluster ID: ae9a91bf699247d4b3b9b07b4a055a6b
I20260620 11:37:59.850800 8075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:37:59.851920 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:37:59.840471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.5.122.190:36739)
0620 11:37:59.840636 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 11:37:59.840644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:37:59.840670 (+ 26us) server_negotiation.cc:407] Connection header received
0620 11:37:59.841202 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:37:59.841209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:37:59.841268 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:37:59.841416 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:37:59.842181 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:59.843055 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:59.843840 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:37:59.844485 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:37:59.844636 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:37:59.847864 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:37:59.847869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:37:59.847869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:37:59.847882 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:37:59.850042 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:59.850467 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:59.850471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:59.850471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:59.850584 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:37:59.850828 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:37:59.850829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:37:59.850829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:37:59.851149 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:37:59.851320 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 11:37:59.851472 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 11:37:59.851637 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260620 11:37:59.857764 8075 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:37:59.858608 8075 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:37:59.865823 8075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45a2561cc7514bd1b0cfa3cf83f4d67d: Generated new TSK 0
I20260620 11:37:59.866120 8075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:37:59.892372 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36739
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:37:59.978026 8087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:37:59.978262 8087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:37:59.978289 8087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:37:59.979496 8087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:37:59.979539 8087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:37:59.979566 8087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:37:59.981015 8087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:37:59.981225 8087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:37:59.981417 8087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:37:59.986533 8093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:59.986636 8095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:37:59.986572 8092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:37:59.986760 8087 server_base.cc:1034] running on GCE node
I20260620 11:37:59.986985 8087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:37:59.987484 8087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:37:59.988680 8087 hybrid_clock.cc:648] HybridClock initialized: now 1781955479988663 us; error 31 us; skew 500 ppm
Jun 20 11:37:59 dist-test-slave-szdr krb5kdc[7980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955479, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:37:59.991204 8087 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:37:59.991664 8087 webserver.cc:460] Webserver started at http://127.5.122.129:46687/ using document root <none> and password file <none>
I20260620 11:37:59.991865 8087 fs_manager.cc:359] Metadata directory not provided
I20260620 11:37:59.991906 8087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:37:59.991997 8087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:37:59.993037 8087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "32d052be7e8c4d968a724c248f2eaf42"
format_stamp: "Formatted at 2026-06-20 11:37:59 on dist-test-slave-szdr"
server_key: "ee023812c0cd7a46dd2f67bfdc9416db"
server_key_iv: "d1fe38d612b8f6b7fabe807c32bbaf0c"
server_key_version: "encryptionkey@0"
I20260620 11:37:59.993341 8087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "32d052be7e8c4d968a724c248f2eaf42"
format_stamp: "Formatted at 2026-06-20 11:37:59 on dist-test-slave-szdr"
server_key: "ee023812c0cd7a46dd2f67bfdc9416db"
server_key_iv: "d1fe38d612b8f6b7fabe807c32bbaf0c"
server_key_version: "encryptionkey@0"
I20260620 11:37:59.994731 8087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:37:59.995520 8102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:37:59.995689 8087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:37:59.995761 8087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "32d052be7e8c4d968a724c248f2eaf42"
format_stamp: "Formatted at 2026-06-20 11:37:59 on dist-test-slave-szdr"
server_key: "ee023812c0cd7a46dd2f67bfdc9416db"
server_key_iv: "d1fe38d612b8f6b7fabe807c32bbaf0c"
server_key_version: "encryptionkey@0"
I20260620 11:37:59.995817 8087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:00.010957 8087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:00.013064 8087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:00.013190 8087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:00.013382 8087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:00.013693 8087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:00.013727 8087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.013777 8087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:00.013803 8087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.019881 8087 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:43273
I20260620 11:38:00.019920 8215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:43273 every 8 connection(s)
I20260620 11:38:00.020262 8087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[7980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955479, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:00.027758 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8087
I20260620 11:38:00.027876 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:00.028055 5610 external_mini_cluster.cc:1428] Setting key c4281238eae7506cf7054d95f6be3cf1
I20260620 11:38:00.029613 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:00.020958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:49589 (local address 127.5.122.190:36739)
0620 11:38:00.021006 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:00.021009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:00.021194 (+ 185us) server_negotiation.cc:407] Connection header received
0620 11:38:00.021674 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:00.021675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:00.021686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:00.021736 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:00.022740 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.023249 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.024243 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.024443 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.024493 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:00.026677 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:00.026678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:00.026678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:00.026686 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:00.028076 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:00.028524 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:00.028526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:00.028526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:00.028590 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:00.028842 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:00.028843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:00.028843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:00.028959 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:00.029035 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:00.029347 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 11:38:00.029465 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 11:38:00.029691 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:00.020941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:36739 (local address 127.5.122.129:49589)
0620 11:38:00.021077 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:00.021120 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:00.021588 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:00.021794 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:00.021797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:00.022446 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:00.022654 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:00.022659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.023338 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.023339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:00.024165 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:00.024168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.024515 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.024515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:00.024616 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:00.025074 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:00.025082 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:00.026585 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:00.028184 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:00.028192 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:00.028194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:00.028440 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:00.028662 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:00.028663 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:00.028663 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:00.028773 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:00.029025 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:00.029032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:00.029176 (+ 144us) client_negotiation.cc:769] Sending connection context
0620 11:38:00.029304 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 11:38:00.029495 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260620 11:38:00.030222 8216 heartbeater.cc:347] Connected to a master server at 127.5.122.190:36739
I20260620 11:38:00.030347 8216 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:00.030531 8216 heartbeater.cc:511] Master 127.5.122.190:36739 requested a full tablet report, sending...
I20260620 11:38:00.031073 8025 ts_manager.cc:194] Registered new tserver with Master: 32d052be7e8c4d968a724c248f2eaf42 (127.5.122.129:43273)
I20260620 11:38:00.031767 8025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:49589
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:38:00.055099 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36739
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:00.143527 8221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:00.143746 8221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:00.143769 8221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:00.144958 8221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:00.144999 8221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:00.145025 8221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:00.146624 8221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:00.146843 8221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:00.147066 8221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:00.152151 8229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:00.152218 8226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:00.152375 8221 server_base.cc:1034] running on GCE node
W20260620 11:38:00.152220 8227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:00.152652 8221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:00.152897 8221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:00.154016 8221 hybrid_clock.cc:648] HybridClock initialized: now 1781955480154001 us; error 40 us; skew 500 ppm
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[7980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955480, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:00.156586 8221 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:38:00.157008 8221 webserver.cc:460] Webserver started at http://127.5.122.130:33557/ using document root <none> and password file <none>
I20260620 11:38:00.157245 8221 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:00.157317 8221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:00.157459 8221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:00.158595 8221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "8f652d77870b45ac8356bec392f84b72"
format_stamp: "Formatted at 2026-06-20 11:38:00 on dist-test-slave-szdr"
server_key: "b012d76140bd49c60eb005457cc6cdea"
server_key_iv: "0b4e68fd7d0ffefc850887f83b080e14"
server_key_version: "encryptionkey@0"
I20260620 11:38:00.158962 8221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "8f652d77870b45ac8356bec392f84b72"
format_stamp: "Formatted at 2026-06-20 11:38:00 on dist-test-slave-szdr"
server_key: "b012d76140bd49c60eb005457cc6cdea"
server_key_iv: "0b4e68fd7d0ffefc850887f83b080e14"
server_key_version: "encryptionkey@0"
I20260620 11:38:00.160282 8221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:00.161013 8236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.161260 8221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:00.161360 8221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "8f652d77870b45ac8356bec392f84b72"
format_stamp: "Formatted at 2026-06-20 11:38:00 on dist-test-slave-szdr"
server_key: "b012d76140bd49c60eb005457cc6cdea"
server_key_iv: "0b4e68fd7d0ffefc850887f83b080e14"
server_key_version: "encryptionkey@0"
I20260620 11:38:00.161435 8221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:00.174746 8221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:00.177135 8221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:00.177296 8221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:00.177532 8221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:00.177840 8221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:00.177873 8221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.177913 8221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:00.177942 8221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.184113 8221 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:45241
I20260620 11:38:00.184182 8349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:45241 every 8 connection(s)
I20260620 11:38:00.184509 8221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:00.190490 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8221
I20260620 11:38:00.190584 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:00.190789 5610 external_mini_cluster.cc:1428] Setting key 9a38fd4b6a9763ec249a2f6f56ece7c0
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[7980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955480, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:00.194900 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:00.185299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:52447 (local address 127.5.122.190:36739)
0620 11:38:00.185365 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:00.185367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:00.185549 (+ 182us) server_negotiation.cc:407] Connection header received
0620 11:38:00.186162 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:00.186165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:00.186179 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:00.186239 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:00.187218 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.187761 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.188807 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.189036 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.189118 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:00.191559 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:00.191560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:00.191561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:00.191570 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:00.193228 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:00.193758 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:00.193759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:00.193760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:00.193823 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:00.194133 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:00.194134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:00.194134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:00.194246 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:00.194316 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:00.194751 (+ 435us) server_negotiation.cc:299] Negotiation successful
0620 11:38:00.194829 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 11:38:00.195204 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:00.185259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:36739 (local address 127.5.122.130:52447)
0620 11:38:00.185414 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:00.185458 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:00.186041 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:00.186316 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:00.186320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:00.186906 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:00.187106 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:00.187110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.187858 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.187859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:00.188696 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:00.188698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.189122 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.189124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:00.189256 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:00.189810 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:00.189820 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:00.191461 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:00.193331 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:00.193339 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:00.193342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:00.193682 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:00.193889 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:00.193890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:00.193891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:00.194072 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:00.194400 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:00.194406 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:00.194593 (+ 187us) client_negotiation.cc:769] Sending connection context
0620 11:38:00.194720 (+ 127us) client_negotiation.cc:240] Negotiation successful
0620 11:38:00.194909 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260620 11:38:00.195674 8350 heartbeater.cc:347] Connected to a master server at 127.5.122.190:36739
I20260620 11:38:00.195792 8350 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:00.196013 8350 heartbeater.cc:511] Master 127.5.122.190:36739 requested a full tablet report, sending...
I20260620 11:38:00.196502 8025 ts_manager.cc:194] Registered new tserver with Master: 8f652d77870b45ac8356bec392f84b72 (127.5.122.130:45241)
I20260620 11:38:00.196923 8025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:52447
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:38:00.218998 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36739
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:00.303759 8355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:00.303995 8355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:00.304023 8355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:00.305346 8355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:00.305399 8355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:00.305469 8355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:00.306994 8355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:00.307245 8355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:00.307487 8355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:00.312744 8361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:00.312916 8363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:00.312856 8355 server_base.cc:1034] running on GCE node
W20260620 11:38:00.312783 8360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:00.313220 8355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:00.313467 8355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:00.314617 8355 hybrid_clock.cc:648] HybridClock initialized: now 1781955480314595 us; error 42 us; skew 500 ppm
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[7980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955480, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:00.317209 8355 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:38:00.317672 8355 webserver.cc:460] Webserver started at http://127.5.122.131:37801/ using document root <none> and password file <none>
I20260620 11:38:00.317893 8355 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:00.317936 8355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:00.318101 8355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:00.319166 8355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "a9d1b42350b74c208763a30b7329ead0"
format_stamp: "Formatted at 2026-06-20 11:38:00 on dist-test-slave-szdr"
server_key: "77a972c73845eb021f007866de6c22f9"
server_key_iv: "aa8f6bada28b88494fcbd4041cf6f890"
server_key_version: "encryptionkey@0"
I20260620 11:38:00.319464 8355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "a9d1b42350b74c208763a30b7329ead0"
format_stamp: "Formatted at 2026-06-20 11:38:00 on dist-test-slave-szdr"
server_key: "77a972c73845eb021f007866de6c22f9"
server_key_iv: "aa8f6bada28b88494fcbd4041cf6f890"
server_key_version: "encryptionkey@0"
I20260620 11:38:00.320739 8355 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:00.321468 8370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.321643 8355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:00.321746 8355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "a9d1b42350b74c208763a30b7329ead0"
format_stamp: "Formatted at 2026-06-20 11:38:00 on dist-test-slave-szdr"
server_key: "77a972c73845eb021f007866de6c22f9"
server_key_iv: "aa8f6bada28b88494fcbd4041cf6f890"
server_key_version: "encryptionkey@0"
I20260620 11:38:00.321830 8355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:00.354174 8355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:00.356750 8355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:00.356941 8355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:00.357203 8355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:00.357576 8355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:00.357614 8355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.357637 8355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:00.357650 8355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:00.364045 8355 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:33257
I20260620 11:38:00.364086 8483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:33257 every 8 connection(s)
I20260620 11:38:00.364473 8355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:00.364666 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8355
I20260620 11:38:00.364758 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:00.364995 5610 external_mini_cluster.cc:1428] Setting key 5d8358ed126fc128352a524cf44608d3
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[7980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955480, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:00.374809 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:00.365481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:54143 (local address 127.5.122.190:36739)
0620 11:38:00.365546 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:00.365549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:00.365847 (+ 298us) server_negotiation.cc:407] Connection header received
0620 11:38:00.366454 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:00.366455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:00.366467 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:00.366525 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:00.367530 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.368069 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.369130 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.369350 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.369436 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:00.371944 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:00.371946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:00.371946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:00.371955 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:00.373241 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:00.373716 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:00.373718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:00.373718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:00.373786 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:00.374117 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:00.374118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:00.374118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:00.374251 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:00.374328 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:00.374624 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 11:38:00.374711 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 11:38:00.375054 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:00.365446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:36739 (local address 127.5.122.131:54143)
0620 11:38:00.365675 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:00.365726 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:00.366343 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:00.366597 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:00.366601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:00.367198 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:00.367416 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:00.367421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.368169 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.368170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:00.369031 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:00.369034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:00.369449 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:00.369451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:00.369591 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:00.370352 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:00.370361 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:00.371835 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:00.373354 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:00.373363 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:00.373366 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:00.373633 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:00.373854 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:00.373855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:00.373855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:00.374037 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:00.374317 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:00.374324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:00.374481 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 11:38:00.374590 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 11:38:00.374772 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":140,"threads_started":1}
I20260620 11:38:00.375581 8484 heartbeater.cc:347] Connected to a master server at 127.5.122.190:36739
I20260620 11:38:00.375691 8484 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:00.375892 8484 heartbeater.cc:511] Master 127.5.122.190:36739 requested a full tablet report, sending...
I20260620 11:38:00.376322 8025 ts_manager.cc:194] Registered new tserver with Master: a9d1b42350b74c208763a30b7329ead0 (127.5.122.131:33257)
I20260620 11:38:00.376852 8025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:54143
I20260620 11:38:00.377131 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 11:38:00.388929 8064 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:00.387851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.5.122.190:36739)
0620 11:38:00.387929 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:00.387933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:00.388037 (+ 104us) server_negotiation.cc:407] Connection header received
0620 11:38:00.388091 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:00.388092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:00.388102 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:00.388155 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:00.388828 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47708: BlockingRecv error: recv got EOF from 127.0.0.1:47708 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 11:38:00.389710 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8087
I20260620 11:38:00.394275 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8221
I20260620 11:38:00.398860 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8355
I20260620 11:38:00.403131 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 7993
2026-06-20T11:38:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1859 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[8496](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[8496](info): set up 2 sockets
Jun 20 11:38:00 dist-test-slave-szdr krb5kdc[8496](info): commencing operation
krb5kdc: starting...
W20260620 11:38:01.445888 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:01 dist-test-slave-szdr krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:01.572563 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41911
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:38231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:01.656141 8509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:01.656477 8509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:01.656523 8509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:01.657882 8509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:01.657979 8509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:01.658043 8509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:01.658071 8509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:01.658090 8509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:01.659859 8509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:38231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:01.660205 8509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:01.660480 8509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:01.665921 8514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:01.665921 8515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:01.665921 8517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:01.666260 8509 server_base.cc:1034] running on GCE node
I20260620 11:38:01.666576 8509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:01.666929 8509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:01.668124 8509 hybrid_clock.cc:648] HybridClock initialized: now 1781955481668112 us; error 60 us; skew 500 ppm
Jun 20 11:38:01 dist-test-slave-szdr krb5kdc[8496](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955481, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:01.671042 8509 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:01.671505 8509 webserver.cc:460] Webserver started at http://127.5.122.190:40865/ using document root <none> and password file <none>
I20260620 11:38:01.671777 8509 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:01.671828 8509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:01.672012 8509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:01.673192 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "01e82b2275f74a58b7f683640816a846"
format_stamp: "Formatted at 2026-06-20 11:38:01 on dist-test-slave-szdr"
server_key: "6afa14f6a290c2f6860d617d33ce68b9"
server_key_iv: "de701edc9fa92c026804eb396b454614"
server_key_version: "encryptionkey@0"
I20260620 11:38:01.673610 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "01e82b2275f74a58b7f683640816a846"
format_stamp: "Formatted at 2026-06-20 11:38:01 on dist-test-slave-szdr"
server_key: "6afa14f6a290c2f6860d617d33ce68b9"
server_key_iv: "de701edc9fa92c026804eb396b454614"
server_key_version: "encryptionkey@0"
I20260620 11:38:01.675164 8509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:01.676026 8524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:01.676260 8509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:01.676374 8509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "01e82b2275f74a58b7f683640816a846"
format_stamp: "Formatted at 2026-06-20 11:38:01 on dist-test-slave-szdr"
server_key: "6afa14f6a290c2f6860d617d33ce68b9"
server_key_iv: "de701edc9fa92c026804eb396b454614"
server_key_version: "encryptionkey@0"
I20260620 11:38:01.676456 8509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:01.699369 8509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:01.701645 8509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:01.701820 8509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:01.705847 8509 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:41911
I20260620 11:38:01.705932 8576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:41911 every 8 connection(s)
I20260620 11:38:01.706360 8509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:01.707309 8577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:01.708134 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8509
I20260620 11:38:01.708252 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:01.708465 5610 external_mini_cluster.cc:1428] Setting key 40d03edc88bae8dcac274b5719e44293
I20260620 11:38:01.710754 8577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846: Bootstrap starting.
I20260620 11:38:01.711557 8577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:01.711869 8577 log.cc:824] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:01.712634 8577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846: No bootstrap required, opened a new log
I20260620 11:38:01.715106 8577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e82b2275f74a58b7f683640816a846" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41911 } }
I20260620 11:38:01.715255 8577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:01.715278 8577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e82b2275f74a58b7f683640816a846, State: Initialized, Role: FOLLOWER
I20260620 11:38:01.715451 8577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e82b2275f74a58b7f683640816a846" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41911 } }
I20260620 11:38:01.715543 8577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:01.715607 8577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:01.715659 8577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:01.716336 8577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e82b2275f74a58b7f683640816a846" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41911 } }
I20260620 11:38:01.716430 8577 leader_election.cc:302] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e82b2275f74a58b7f683640816a846; no voters:
I20260620 11:38:01.716637 8577 leader_election.cc:288] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:01.716682 8582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:01.716863 8582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [term 1 LEADER]: Becoming Leader. State: Replica: 01e82b2275f74a58b7f683640816a846, State: Running, Role: LEADER
I20260620 11:38:01.716957 8577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:01.716995 8582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e82b2275f74a58b7f683640816a846" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41911 } }
I20260620 11:38:01.717417 8583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e82b2275f74a58b7f683640816a846. Latest consensus state: current_term: 1 leader_uuid: "01e82b2275f74a58b7f683640816a846" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e82b2275f74a58b7f683640816a846" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41911 } } }
I20260620 11:38:01.717406 8584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e82b2275f74a58b7f683640816a846" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e82b2275f74a58b7f683640816a846" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41911 } } }
I20260620 11:38:01.717504 8583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:01.717511 8584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:01.717905 8588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:01.718420 8588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:01.720481 8588 catalog_manager.cc:1347] Generated new cluster ID: 7fb41035ee2d492881187a5a0b6b00e3
I20260620 11:38:01.720542 8588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:38:01 dist-test-slave-szdr krb5kdc[8496](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:01.725162 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:01.709063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.5.122.190:41911)
0620 11:38:01.709281 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:01.709291 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:01.709326 (+ 35us) server_negotiation.cc:407] Connection header received
0620 11:38:01.709929 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:01.709939 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:01.710035 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:01.710195 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:01.710868 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:01.712259 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:01.713124 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:01.713773 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:01.713904 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:01.721528 (+ 7624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:01.721533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:01.721533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:01.721546 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:01.723503 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:01.723919 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:01.723922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:01.723922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:01.724022 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:01.724262 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:01.724263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:01.724263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:01.724524 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:01.724714 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:01.724807 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 11:38:01.724960 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260620 11:38:01.726251 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:38:01.743844 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:01.739179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.5.122.190:41911)
0620 11:38:01.739227 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:01.739229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:01.739352 (+ 123us) server_negotiation.cc:407] Connection header received
0620 11:38:01.739406 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:01.739407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:01.739418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:01.739492 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:01.740021 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:01.740768 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:01.741932 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:01.742310 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:01.742375 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:01.743238 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:01.743240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:01.743240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:01.743496 (+ 256us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:01.743518 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:01.743526 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:01.743583 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:01.743725 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 11:38:01.743755 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:38:01.753014 8588 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:01.753607 8588 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:01.757725 8588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e82b2275f74a58b7f683640816a846: Generated new TSK 0
I20260620 11:38:01.757952 8588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:38:01.760998 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8509
2026-06-20T11:38:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1356 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:01 dist-test-slave-szdr krb5kdc[8610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:01 dist-test-slave-szdr krb5kdc[8610](info): set up 2 sockets
Jun 20 11:38:01 dist-test-slave-szdr krb5kdc[8610](info): commencing operation
krb5kdc: starting...
W20260620 11:38:02.801863 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:02 dist-test-slave-szdr krb5kdc[8610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:02.930227 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41697
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:03.014518 8623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:03.014860 8623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:03.014905 8623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:03.016248 8623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:03.016353 8623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:03.016378 8623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:03.016402 8623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:03.016429 8623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:03.018359 8623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:03.018859 8623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:03.019167 8623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:03.025277 8628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:03.025277 8629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:03.025285 8631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:03.025437 8623 server_base.cc:1034] running on GCE node
I20260620 11:38:03.025843 8623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:03.026218 8623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:03.027423 8623 hybrid_clock.cc:648] HybridClock initialized: now 1781955483027400 us; error 42 us; skew 500 ppm
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:03.030529 8623 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:03.031152 8623 webserver.cc:460] Webserver started at http://127.5.122.190:40497/ using document root <none> and password file <none>
I20260620 11:38:03.031469 8623 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:03.031550 8623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:03.031710 8623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:03.033336 8623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "19a1fbb94956437c8a0fb71c24706829"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "013985030d14ff7a972387b011aebbc0"
server_key_iv: "237285c9ce5926db6d309ee988c2b2de"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.033834 8623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "19a1fbb94956437c8a0fb71c24706829"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "013985030d14ff7a972387b011aebbc0"
server_key_iv: "237285c9ce5926db6d309ee988c2b2de"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.035810 8623 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260620 11:38:03.036624 8638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.036883 8623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:03.036998 8623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "19a1fbb94956437c8a0fb71c24706829"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "013985030d14ff7a972387b011aebbc0"
server_key_iv: "237285c9ce5926db6d309ee988c2b2de"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.037106 8623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:03.052157 8623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:03.054253 8623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:03.054375 8623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:03.058351 8623 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:41697
I20260620 11:38:03.058384 8690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:41697 every 8 connection(s)
I20260620 11:38:03.058769 8623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:03.059787 8691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:03.062291 8691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829: Bootstrap starting.
I20260620 11:38:03.062894 8691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:03.063194 8691 log.cc:824] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:03.063973 8691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829: No bootstrap required, opened a new log
I20260620 11:38:03.065629 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8623
I20260620 11:38:03.065742 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:03.065917 5610 external_mini_cluster.cc:1428] Setting key 2b13af29273ed550bd09ad9a3b8491ea
I20260620 11:38:03.065834 8691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a1fbb94956437c8a0fb71c24706829" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41697 } }
I20260620 11:38:03.066057 8691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:03.066094 8691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a1fbb94956437c8a0fb71c24706829, State: Initialized, Role: FOLLOWER
I20260620 11:38:03.066258 8691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a1fbb94956437c8a0fb71c24706829" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41697 } }
I20260620 11:38:03.066371 8691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:03.066432 8691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:03.066493 8691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:03.067425 8691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a1fbb94956437c8a0fb71c24706829" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41697 } }
I20260620 11:38:03.067559 8691 leader_election.cc:302] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a1fbb94956437c8a0fb71c24706829; no voters:
I20260620 11:38:03.067749 8691 leader_election.cc:288] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:03.067790 8696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:03.067983 8696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [term 1 LEADER]: Becoming Leader. State: Replica: 19a1fbb94956437c8a0fb71c24706829, State: Running, Role: LEADER
I20260620 11:38:03.068106 8691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:03.068122 8696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a1fbb94956437c8a0fb71c24706829" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41697 } }
I20260620 11:38:03.068594 8698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a1fbb94956437c8a0fb71c24706829. Latest consensus state: current_term: 1 leader_uuid: "19a1fbb94956437c8a0fb71c24706829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a1fbb94956437c8a0fb71c24706829" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41697 } } }
I20260620 11:38:03.068693 8698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:03.068902 8697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19a1fbb94956437c8a0fb71c24706829" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a1fbb94956437c8a0fb71c24706829" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41697 } } }
I20260620 11:38:03.068948 8704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:03.068962 8697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:03.069666 8704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:03.072618 8704 catalog_manager.cc:1347] Generated new cluster ID: 16172764dbe44002b5894fbf81052e85
I20260620 11:38:03.072669 8704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:03.078761 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.066485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.5.122.190:41697)
0620 11:38:03.066708 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:03.066719 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:03.067809 (+ 1090us) server_negotiation.cc:407] Connection header received
0620 11:38:03.067994 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:03.068002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:03.068065 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:03.069139 (+ 1074us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:03.069328 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.070529 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.071329 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.071979 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.072440 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.074634 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:03.074638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:03.074638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:03.074651 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:03.076937 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.077402 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.077406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.077407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.077520 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.077749 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.077750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.077750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.078093 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:03.078279 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:03.078358 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 11:38:03.078511 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260620 11:38:03.088979 8704 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:03.089573 8704 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:03.093834 8704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19a1fbb94956437c8a0fb71c24706829: Generated new TSK 0
I20260620 11:38:03.094070 8704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:38:03.126176 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41697
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:03.211735 8717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:03.211959 8717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:03.211983 8717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:03.213207 8717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:03.213261 8717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:03.213289 8717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:03.214847 8717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:03.215109 8717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:03.215387 8717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:03.220767 8722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:03.220796 8723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:03.220796 8717 server_base.cc:1034] running on GCE node
W20260620 11:38:03.221130 8725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:03.221407 8717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:03.221680 8717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:03.222831 8717 hybrid_clock.cc:648] HybridClock initialized: now 1781955483222819 us; error 34 us; skew 500 ppm
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:03.225528 8717 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:38:03.226091 8717 webserver.cc:460] Webserver started at http://127.5.122.129:46807/ using document root <none> and password file <none>
I20260620 11:38:03.226356 8717 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:03.226410 8717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:03.226572 8717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:03.227720 8717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "1c90132d6f89488caa739aed13f23f0b"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "fc25f59c2c9a4dc97edbff3f2015eb96"
server_key_iv: "97de5e26a25526bebc9f2c7c27d3ffc1"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.228135 8717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "1c90132d6f89488caa739aed13f23f0b"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "fc25f59c2c9a4dc97edbff3f2015eb96"
server_key_iv: "97de5e26a25526bebc9f2c7c27d3ffc1"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.229650 8717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:03.230507 8732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.230696 8717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:38:03.230808 8717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "1c90132d6f89488caa739aed13f23f0b"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "fc25f59c2c9a4dc97edbff3f2015eb96"
server_key_iv: "97de5e26a25526bebc9f2c7c27d3ffc1"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.230901 8717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:03.240470 8717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:03.242909 8717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:03.243088 8717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:03.243352 8717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:03.243691 8717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:03.243753 8717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.243798 8717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:03.243818 8717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.249838 8717 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:43591
I20260620 11:38:03.249874 8845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:43591 every 8 connection(s)
I20260620 11:38:03.250249 8717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:03.251996 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8717
I20260620 11:38:03.252152 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:03.252346 5610 external_mini_cluster.cc:1428] Setting key d60fdfb606b067e354f1d5150a3fc1bc
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
I20260620 11:38:03.262051 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.251194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:59951 (local address 127.5.122.190:41697)
0620 11:38:03.251262 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:03.251264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:03.251519 (+ 255us) server_negotiation.cc:407] Connection header received
0620 11:38:03.252089 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:03.252091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:03.252105 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:03.252176 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:03.253501 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.254086 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.255700 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.255920 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.255990 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.258812 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:03.258813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:03.258814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:03.258823 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:03.260341 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.260848 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.260849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.260850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.260922 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.261273 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.261274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.261274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.261413 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:03.261504 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:03.261824 (+ 320us) server_negotiation.cc:299] Negotiation successful
0620 11:38:03.261924 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 11:38:03.262301 8848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.251166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41697 (local address 127.5.122.129:59951)
0620 11:38:03.251382 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:03.251422 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:03.251964 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:03.252258 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:03.252262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:03.253136 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:03.253380 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:03.253386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.254186 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.254187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:03.255580 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:03.255584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.256033 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.256034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:03.256207 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.256830 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:03.256843 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:03.258678 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:03.260483 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:03.260493 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:03.260496 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:03.260754 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:03.261013 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:03.261014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:03.261014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:03.261186 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:03.261501 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:03.261509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:03.261668 (+ 159us) client_negotiation.cc:769] Sending connection context
0620 11:38:03.261799 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 11:38:03.262021 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260620 11:38:03.262804 8846 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41697
I20260620 11:38:03.262941 8846 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:03.263175 8846 heartbeater.cc:511] Master 127.5.122.190:41697 requested a full tablet report, sending...
I20260620 11:38:03.263872 8655 ts_manager.cc:194] Registered new tserver with Master: 1c90132d6f89488caa739aed13f23f0b (127.5.122.129:43591)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
I20260620 11:38:03.264832 8655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:59951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:38:03.281364 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41697
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:03.363481 8851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:03.363716 8851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:03.363744 8851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:03.365003 8851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:03.365061 8851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:03.365091 8851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:03.366724 8851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:03.367026 8851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:03.367307 8851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:03.373401 8857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:03.373420 8859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:03.373438 8851 server_base.cc:1034] running on GCE node
W20260620 11:38:03.373382 8856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:03.374048 8851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:03.374331 8851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:03.375489 8851 hybrid_clock.cc:648] HybridClock initialized: now 1781955483375476 us; error 33 us; skew 500 ppm
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:03.378290 8851 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:38:03.378831 8851 webserver.cc:460] Webserver started at http://127.5.122.130:40149/ using document root <none> and password file <none>
I20260620 11:38:03.379104 8851 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:03.379153 8851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:03.379328 8851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:03.380513 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "547d9bcc736348d8a356b46d23148dfd"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "d77f6b42dde7f83e27e1b8c6d2be75b9"
server_key_iv: "93255ff42f98f6b0a0478962fe77d1bc"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.380894 8851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "547d9bcc736348d8a356b46d23148dfd"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "d77f6b42dde7f83e27e1b8c6d2be75b9"
server_key_iv: "93255ff42f98f6b0a0478962fe77d1bc"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.382409 8851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:03.383251 8866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.383507 8851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:03.383589 8851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "547d9bcc736348d8a356b46d23148dfd"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "d77f6b42dde7f83e27e1b8c6d2be75b9"
server_key_iv: "93255ff42f98f6b0a0478962fe77d1bc"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.383688 8851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:03.403101 8851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:03.405148 8851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:03.405325 8851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:03.405587 8851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:03.405925 8851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:03.406010 8851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.406054 8851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:03.406080 8851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.412379 8851 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:35571
I20260620 11:38:03.412469 8979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:35571 every 8 connection(s)
I20260620 11:38:03.412824 8851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:03.416541 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8851
I20260620 11:38:03.416687 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:03.416908 5610 external_mini_cluster.cc:1428] Setting key fd554168f7cdd2140dcb92ecf8945f93
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:03.424077 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.413710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:52375 (local address 127.5.122.190:41697)
0620 11:38:03.413779 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:03.413805 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:03.414055 (+ 250us) server_negotiation.cc:407] Connection header received
0620 11:38:03.414631 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:03.414632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:03.414645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:03.414700 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:03.415761 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.416274 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.417617 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.417869 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.417932 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.420625 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:03.420627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:03.420627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:03.420636 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:03.422491 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.422956 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.422957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.422958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.423032 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.423294 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.423295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.423296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.423475 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:03.423568 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:03.423854 (+ 286us) server_negotiation.cc:299] Negotiation successful
0620 11:38:03.423955 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 11:38:03.424113 8982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.413681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41697 (local address 127.5.122.130:52375)
0620 11:38:03.413876 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:03.413923 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:03.414509 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:03.414794 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:03.414797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:03.415410 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:03.415629 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:03.415634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.416414 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.416415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:03.417473 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:03.417476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.418000 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.418002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:03.418164 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.418758 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:03.418770 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:03.420504 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:03.422605 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:03.422613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:03.422616 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:03.422867 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:03.423108 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:03.423108 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:03.423109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:03.423219 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:03.423559 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:03.423565 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:03.423700 (+ 135us) client_negotiation.cc:769] Sending connection context
0620 11:38:03.423790 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 11:38:03.423928 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260620 11:38:03.424732 8980 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41697
I20260620 11:38:03.424860 8980 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:03.425096 8980 heartbeater.cc:511] Master 127.5.122.190:41697 requested a full tablet report, sending...
I20260620 11:38:03.425565 8655 ts_manager.cc:194] Registered new tserver with Master: 547d9bcc736348d8a356b46d23148dfd (127.5.122.130:35571)
I20260620 11:38:03.426019 8655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:52375
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:38:03.446249 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41697
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:03.531405 8985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:03.531651 8985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:03.531677 8985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:03.532948 8985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:03.533047 8985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:03.533099 8985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:03.534791 8985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:03.535096 8985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:03.535357 8985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:03.540944 8991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:03.540952 8993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:03.540951 8990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:03.541235 8985 server_base.cc:1034] running on GCE node
I20260620 11:38:03.541476 8985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:03.541757 8985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:03.542940 8985 hybrid_clock.cc:648] HybridClock initialized: now 1781955483542910 us; error 64 us; skew 500 ppm
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:03.545630 8985 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:38:03.546224 8985 webserver.cc:460] Webserver started at http://127.5.122.131:43899/ using document root <none> and password file <none>
I20260620 11:38:03.546478 8985 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:03.546559 8985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:03.546721 8985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:03.547842 8985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "839fce8bd37848d59df882e12cb9d62f"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "33d67f2503d2a01e3d57aaaf2b51551c"
server_key_iv: "10f1fc1f4d29aae587d2996819231876"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.548264 8985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "839fce8bd37848d59df882e12cb9d62f"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "33d67f2503d2a01e3d57aaaf2b51551c"
server_key_iv: "10f1fc1f4d29aae587d2996819231876"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.549728 8985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:03.550582 9000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.550765 8985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.550865 8985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "839fce8bd37848d59df882e12cb9d62f"
format_stamp: "Formatted at 2026-06-20 11:38:03 on dist-test-slave-szdr"
server_key: "33d67f2503d2a01e3d57aaaf2b51551c"
server_key_iv: "10f1fc1f4d29aae587d2996819231876"
server_key_version: "encryptionkey@0"
I20260620 11:38:03.550959 8985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:03.564642 8985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:03.566722 8985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:03.566903 8985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:03.567149 8985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:03.567489 8985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:03.567566 8985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.567610 8985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:03.567631 8985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:03.574306 8985 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:42219
I20260620 11:38:03.574344 9113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:42219 every 8 connection(s)
I20260620 11:38:03.574707 8985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:03.581694 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 8985
I20260620 11:38:03.581776 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:03.582046 5610 external_mini_cluster.cc:1428] Setting key 19fc550f29f88a34177d8085017b7f36
I20260620 11:38:03.584883 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.575515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:41173 (local address 127.5.122.190:41697)
0620 11:38:03.575564 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:03.575566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:03.575803 (+ 237us) server_negotiation.cc:407] Connection header received
0620 11:38:03.576345 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:03.576346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:03.576355 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:03.576417 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:03.577415 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.577920 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.579346 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.579539 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.579597 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.581950 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:03.581952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:03.581952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:03.581984 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:03.583641 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.584042 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.584043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.584043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.584109 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.584332 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.584333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.584333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.584445 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:03.584496 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:03.584746 (+ 250us) server_negotiation.cc:299] Negotiation successful
0620 11:38:03.584815 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:38:03.585116 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.575462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41697 (local address 127.5.122.131:41173)
0620 11:38:03.575659 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:03.575705 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:03.576252 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:03.576503 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:03.576506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:03.577119 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:03.577328 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:03.577333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.578121 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.578122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:03.579263 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:03.579265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.579617 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.579618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:03.579721 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.580298 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:03.580309 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:03.581849 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:03.583721 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:03.583728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:03.583730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:03.583961 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:03.584171 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:03.584171 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:03.584172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:03.584274 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:03.584503 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:03.584507 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:03.584630 (+ 123us) client_negotiation.cc:769] Sending connection context
0620 11:38:03.584717 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 11:38:03.584845 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260620 11:38:03.585655 9114 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41697
I20260620 11:38:03.585772 9114 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:03.586014 9114 heartbeater.cc:511] Master 127.5.122.190:41697 requested a full tablet report, sending...
I20260620 11:38:03.586427 8655 ts_manager.cc:194] Registered new tserver with Master: 839fce8bd37848d59df882e12cb9d62f (127.5.122.131:42219)
I20260620 11:38:03.586908 8655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:41173
I20260620 11:38:03.594148 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[8610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955483, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:03.618283 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:03.610275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.5.122.190:41697)
0620 11:38:03.610368 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:03.610370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:03.610490 (+ 120us) server_negotiation.cc:407] Connection header received
0620 11:38:03.610605 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:03.610606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:03.610617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:03.610698 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:03.611350 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.611929 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.612714 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:03.612919 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:03.613017 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:03.615549 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:03.615550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:03.615551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:03.615564 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:03.616865 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.617296 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.617297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.617297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.617363 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:03.617596 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:03.617597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:03.617597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:03.617764 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:03.617879 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:03.618082 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 11:38:03.618174 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260620 11:38:03.618593 8655 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44562
I20260620 11:38:03.619215 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8717
I20260620 11:38:03.624065 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8851
I20260620 11:38:03.628764 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8985
I20260620 11:38:03.633272 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 8623
2026-06-20T11:38:03Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1874 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[9126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[9126](info): set up 2 sockets
Jun 20 11:38:03 dist-test-slave-szdr krb5kdc[9126](info): commencing operation
krb5kdc: starting...
W20260620 11:38:04.677913 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:04 dist-test-slave-szdr krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:04.806128 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:43755
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:43755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:04.891567 9139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:04.891870 9139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:04.891908 9139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:04.893170 9139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:04.893265 9139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:04.893288 9139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:04.893309 9139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:04.893337 9139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:04.895074 9139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:43755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:43755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:04.895408 9139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:04.895673 9139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:04.901013 9144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:04.901013 9147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:04.901170 9145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:04.901381 9139 server_base.cc:1034] running on GCE node
I20260620 11:38:04.901710 9139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:04.902117 9139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:04.903321 9139 hybrid_clock.cc:648] HybridClock initialized: now 1781955484903300 us; error 41 us; skew 500 ppm
Jun 20 11:38:04 dist-test-slave-szdr krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955484, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:04.905923 9139 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:04.906446 9139 webserver.cc:460] Webserver started at http://127.5.122.190:44551/ using document root <none> and password file <none>
I20260620 11:38:04.906698 9139 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:04.906790 9139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:04.906942 9139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:04.908090 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "b88892f2a9ce405184a6dd143e727edb"
format_stamp: "Formatted at 2026-06-20 11:38:04 on dist-test-slave-szdr"
server_key: "ffc2f3f86bba77a582a7c60f7e3994ea"
server_key_iv: "4531622dd1d71125cfa464290d620531"
server_key_version: "encryptionkey@0"
I20260620 11:38:04.908504 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "b88892f2a9ce405184a6dd143e727edb"
format_stamp: "Formatted at 2026-06-20 11:38:04 on dist-test-slave-szdr"
server_key: "ffc2f3f86bba77a582a7c60f7e3994ea"
server_key_iv: "4531622dd1d71125cfa464290d620531"
server_key_version: "encryptionkey@0"
I20260620 11:38:04.910049 9139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260620 11:38:04.910876 9154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:04.911105 9139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:04.911227 9139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "b88892f2a9ce405184a6dd143e727edb"
format_stamp: "Formatted at 2026-06-20 11:38:04 on dist-test-slave-szdr"
server_key: "ffc2f3f86bba77a582a7c60f7e3994ea"
server_key_iv: "4531622dd1d71125cfa464290d620531"
server_key_version: "encryptionkey@0"
I20260620 11:38:04.911319 9139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:04.919376 9139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:04.921353 9139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:04.921535 9139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:04.925815 9139 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:43755
I20260620 11:38:04.925848 9206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:43755 every 8 connection(s)
I20260620 11:38:04.926441 9139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:04.927165 9207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:04.929518 9207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb: Bootstrap starting.
I20260620 11:38:04.930229 9207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:04.930478 9207 log.cc:824] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:04.931208 9207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb: No bootstrap required, opened a new log
I20260620 11:38:04.931522 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 9139
I20260620 11:38:04.931648 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:04.931835 5610 external_mini_cluster.cc:1428] Setting key d5e8d9d241905d8fa88dec255413bec0
I20260620 11:38:04.933877 9207 raft_consensus.cc:348] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88892f2a9ce405184a6dd143e727edb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 43755 } }
I20260620 11:38:04.934149 9207 raft_consensus.cc:374] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:04.934198 9207 raft_consensus.cc:729] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88892f2a9ce405184a6dd143e727edb, State: Initialized, Role: FOLLOWER
I20260620 11:38:04.934351 9207 consensus_queue.cc:260] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88892f2a9ce405184a6dd143e727edb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 43755 } }
I20260620 11:38:04.934442 9207 raft_consensus.cc:388] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:04.934478 9207 raft_consensus.cc:482] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:04.934533 9207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:04.935276 9207 raft_consensus.cc:504] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88892f2a9ce405184a6dd143e727edb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 43755 } }
I20260620 11:38:04.935371 9207 leader_election.cc:302] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b88892f2a9ce405184a6dd143e727edb; no voters:
I20260620 11:38:04.935567 9207 leader_election.cc:288] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:04.935596 9212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:04.935710 9212 raft_consensus.cc:686] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [term 1 LEADER]: Becoming Leader. State: Replica: b88892f2a9ce405184a6dd143e727edb, State: Running, Role: LEADER
I20260620 11:38:04.935796 9212 consensus_queue.cc:237] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88892f2a9ce405184a6dd143e727edb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 43755 } }
I20260620 11:38:04.935844 9207 sys_catalog.cc:564] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:04.936122 9212 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b88892f2a9ce405184a6dd143e727edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88892f2a9ce405184a6dd143e727edb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 43755 } } }
I20260620 11:38:04.936195 9212 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:04.936318 9212 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b88892f2a9ce405184a6dd143e727edb. Latest consensus state: current_term: 1 leader_uuid: "b88892f2a9ce405184a6dd143e727edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88892f2a9ce405184a6dd143e727edb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 43755 } } }
I20260620 11:38:04.936389 9212 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:04.936694 9216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:04.937268 9216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:04.939257 9216 catalog_manager.cc:1347] Generated new cluster ID: be6bbb38c91d46a69b51a37c784e6783
I20260620 11:38:04.939311 9216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:38:04 dist-test-slave-szdr krb5kdc[9126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:04.944569 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:04.932418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.5.122.190:43755)
0620 11:38:04.932628 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:04.932636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:04.932664 (+ 28us) server_negotiation.cc:407] Connection header received
0620 11:38:04.933855 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:04.933863 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:04.933918 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:04.934124 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:04.934975 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:04.936362 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:04.937537 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:04.938171 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:04.938320 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:04.940868 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:04.940872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:04.940873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:04.940886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:04.942811 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:04.943242 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:04.943246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:04.943246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:04.943362 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:04.943606 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:04.943607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:04.943607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:04.943891 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:04.944068 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:04.944190 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 11:38:04.944347 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260620 11:38:04.947613 9216 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:04.948182 9216 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:04.952240 9216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b88892f2a9ce405184a6dd143e727edb: Generated new TSK 0
I20260620 11:38:04.952436 9216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:38:04.985811 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:43755
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:05.095414 9233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:05.095661 9233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:05.095703 9233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:05.096935 9233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:05.096999 9233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:05.097044 9233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:05.098681 9233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:43755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:05.098958 9233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:05.099220 9233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:05.104705 9238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:05.104713 9239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:05.104713 9241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:05.105176 9233 server_base.cc:1034] running on GCE node
I20260620 11:38:05.105425 9233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:05.105700 9233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:05.106877 9233 hybrid_clock.cc:648] HybridClock initialized: now 1781955485106851 us; error 41 us; skew 500 ppm
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:05.109745 9233 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:38:05.110399 9233 webserver.cc:460] Webserver started at http://127.5.122.129:40385/ using document root <none> and password file <none>
I20260620 11:38:05.110671 9233 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:05.110747 9233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:05.110915 9233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:05.112084 9233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "252003b3a4df462ab4966b1ade5e6f6e"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "0ce180e995273f2a26c3ab6cbbc786fe"
server_key_iv: "aa6e0b5a3902faa4d03208edc00f2738"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.112478 9233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "252003b3a4df462ab4966b1ade5e6f6e"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "0ce180e995273f2a26c3ab6cbbc786fe"
server_key_iv: "aa6e0b5a3902faa4d03208edc00f2738"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.113906 9233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:05.114811 9248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.115003 9233 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:05.115091 9233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "252003b3a4df462ab4966b1ade5e6f6e"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "0ce180e995273f2a26c3ab6cbbc786fe"
server_key_iv: "aa6e0b5a3902faa4d03208edc00f2738"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.115199 9233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:05.124302 9233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:05.126976 9233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:05.127143 9233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:05.127429 9233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:05.127760 9233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:05.127795 9233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.127820 9233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:05.127859 9233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.134361 9233 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:40847
I20260620 11:38:05.134447 9361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:40847 every 8 connection(s)
I20260620 11:38:05.134783 9233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:05.142352 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 9233
I20260620 11:38:05.142464 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:05.142674 5610 external_mini_cluster.cc:1428] Setting key 26cbaac3bf0d15000ce9814691edacd4
I20260620 11:38:05.146471 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.135683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:38813 (local address 127.5.122.190:43755)
0620 11:38:05.135762 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:05.135764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:05.136027 (+ 263us) server_negotiation.cc:407] Connection header received
0620 11:38:05.137241 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:05.137243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:05.137257 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:05.137325 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:05.138125 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.138645 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.139768 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.139979 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.140034 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.142822 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:05.142824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:05.142825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:05.142833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:05.144572 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.145180 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.145182 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.145182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.145263 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.145556 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.145557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.145557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.145735 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:05.145821 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:05.146184 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 11:38:05.146308 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 11:38:05.146483 9364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.135630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:43755 (local address 127.5.122.129:38813)
0620 11:38:05.135870 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:05.135937 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:05.137096 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:05.137392 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:05.137396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:05.137757 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:05.138039 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:05.138044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.138738 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.138740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:05.139675 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:05.139678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.140069 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.140070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:05.140194 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.140779 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:05.140787 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:05.142662 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:05.144676 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:05.144688 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:05.144691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:05.145088 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:05.145354 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:05.145355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:05.145355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:05.145482 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:05.145816 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:05.145821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:05.145957 (+ 136us) client_negotiation.cc:769] Sending connection context
0620 11:38:05.146133 (+ 176us) client_negotiation.cc:240] Negotiation successful
0620 11:38:05.146282 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":147,"threads_started":1}
I20260620 11:38:05.147459 9362 heartbeater.cc:347] Connected to a master server at 127.5.122.190:43755
I20260620 11:38:05.147596 9362 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:05.147861 9362 heartbeater.cc:511] Master 127.5.122.190:43755 requested a full tablet report, sending...
I20260620 11:38:05.148481 9171 ts_manager.cc:194] Registered new tserver with Master: 252003b3a4df462ab4966b1ade5e6f6e (127.5.122.129:40847)
I20260620 11:38:05.149120 9171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:38813
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:38:05.173702 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:43755
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:05.258471 9367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:05.258816 9367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:05.258846 9367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:05.260102 9367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:05.260149 9367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:05.260178 9367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:05.261732 9367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:43755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:05.261950 9367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:05.262257 9367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:05.267591 9375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:05.267585 9372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:05.267591 9373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:05.267796 9367 server_base.cc:1034] running on GCE node
I20260620 11:38:05.268038 9367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:05.268270 9367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:05.269423 9367 hybrid_clock.cc:648] HybridClock initialized: now 1781955485269404 us; error 50 us; skew 500 ppm
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:05.272125 9367 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:38:05.272680 9367 webserver.cc:460] Webserver started at http://127.5.122.130:33967/ using document root <none> and password file <none>
I20260620 11:38:05.272929 9367 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:05.273007 9367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:05.273167 9367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:05.274354 9367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "9f0bc1c89a3b4665bba674eb01f27d3e"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "41dc11355d00d4a5ca75f0bc48cd68a0"
server_key_iv: "5b73d021e9ef04450d65449b87f8719a"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.274772 9367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "9f0bc1c89a3b4665bba674eb01f27d3e"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "41dc11355d00d4a5ca75f0bc48cd68a0"
server_key_iv: "5b73d021e9ef04450d65449b87f8719a"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.276228 9367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:05.277037 9382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.277275 9367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:05.277390 9367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "9f0bc1c89a3b4665bba674eb01f27d3e"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "41dc11355d00d4a5ca75f0bc48cd68a0"
server_key_iv: "5b73d021e9ef04450d65449b87f8719a"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.277473 9367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:05.285836 9367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:05.287876 9367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:05.288074 9367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:05.288347 9367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:05.288697 9367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:05.288759 9367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.288812 9367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:05.288837 9367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.295125 9367 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:44961
I20260620 11:38:05.295177 9495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:44961 every 8 connection(s)
I20260620 11:38:05.295563 9367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:05.299401 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 9367
I20260620 11:38:05.299552 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:05.299736 5610 external_mini_cluster.cc:1428] Setting key 6bf63b1f772afe8fe05fda9662e7428a
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:05.306897 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.296430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:46739 (local address 127.5.122.190:43755)
0620 11:38:05.296513 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:05.296516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:05.296727 (+ 211us) server_negotiation.cc:407] Connection header received
0620 11:38:05.297733 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:05.297735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:05.297747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:05.297811 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:05.298612 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.299128 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.300636 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.300945 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.301023 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.303573 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:05.303574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:05.303575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:05.303584 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:05.305217 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.305751 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.305752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.305753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.305814 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.306171 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.306172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.306173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.306304 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:05.306390 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:05.306702 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 11:38:05.306798 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 11:38:05.307147 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.296404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:43755 (local address 127.5.122.130:46739)
0620 11:38:05.296579 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:05.296619 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:05.297609 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:05.297911 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:05.297915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:05.298295 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:05.298519 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:05.298524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.299274 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.299274 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:05.300488 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:05.300491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.301040 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.301041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:05.301156 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.301805 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:05.301815 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:05.303454 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:05.305373 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:05.305383 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:05.305386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:05.305648 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:05.305894 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:05.305895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:05.305895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:05.306094 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:05.306379 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:05.306386 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:05.306548 (+ 162us) client_negotiation.cc:769] Sending connection context
0620 11:38:05.306668 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 11:38:05.306862 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260620 11:38:05.307708 9496 heartbeater.cc:347] Connected to a master server at 127.5.122.190:43755
I20260620 11:38:05.307838 9496 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:05.308073 9496 heartbeater.cc:511] Master 127.5.122.190:43755 requested a full tablet report, sending...
I20260620 11:38:05.308568 9171 ts_manager.cc:194] Registered new tserver with Master: 9f0bc1c89a3b4665bba674eb01f27d3e (127.5.122.130:44961)
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
I20260620 11:38:05.309110 9171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:46739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:38:05.331419 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:43755
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:45129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:05.418272 9501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:05.418572 9501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:05.418599 9501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:05.419842 9501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:05.419894 9501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:05.419924 9501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:05.421543 9501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:43755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:05.421842 9501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:05.422144 9501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:05.427539 9509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:05.427574 9501 server_base.cc:1034] running on GCE node
W20260620 11:38:05.427539 9506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:05.427539 9507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:05.428074 9501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:05.428297 9501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:05.429441 9501 hybrid_clock.cc:648] HybridClock initialized: now 1781955485429416 us; error 43 us; skew 500 ppm
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:05.432392 9501 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:38:05.432898 9501 webserver.cc:460] Webserver started at http://127.5.122.131:36455/ using document root <none> and password file <none>
I20260620 11:38:05.433172 9501 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:05.433251 9501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:05.433409 9501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:05.434582 9501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "f101711f035d4c5dbded91e10bb495d0"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "7e77abe7017a433ac7c7705b30717fde"
server_key_iv: "239ad4faa361a40aafb89d744a520985"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.434989 9501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "f101711f035d4c5dbded91e10bb495d0"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "7e77abe7017a433ac7c7705b30717fde"
server_key_iv: "239ad4faa361a40aafb89d744a520985"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.436657 9501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:05.437568 9516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.437790 9501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:05.437919 9501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "f101711f035d4c5dbded91e10bb495d0"
format_stamp: "Formatted at 2026-06-20 11:38:05 on dist-test-slave-szdr"
server_key: "7e77abe7017a433ac7c7705b30717fde"
server_key_iv: "239ad4faa361a40aafb89d744a520985"
server_key_version: "encryptionkey@0"
I20260620 11:38:05.438024 9501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:05.447575 9501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:05.449538 9501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:05.449724 9501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:05.450034 9501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:05.450410 9501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:05.450472 9501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.450515 9501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:05.450542 9501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:05.456573 9501 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:44867
I20260620 11:38:05.456653 9629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:44867 every 8 connection(s)
I20260620 11:38:05.457002 9501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:05.457636 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 9501
I20260620 11:38:05.457738 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:05.457950 5610 external_mini_cluster.cc:1428] Setting key 545d81cd2b506910eded5a711a5b55f4
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:05.467202 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.457732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:39031 (local address 127.5.122.190:43755)
0620 11:38:05.457782 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:05.457784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:05.458072 (+ 288us) server_negotiation.cc:407] Connection header received
0620 11:38:05.459341 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:05.459342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:05.459352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:05.459402 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:05.460014 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.460527 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.461753 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.461940 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.462078 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.464395 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:05.464396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:05.464397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:05.464404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:05.465904 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.466363 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.466364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.466365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.466421 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.466648 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.466649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.466649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.466758 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:05.466808 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:05.467067 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 11:38:05.467133 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:38:05.467374 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.457737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:43755 (local address 127.5.122.131:39031)
0620 11:38:05.457922 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:05.457980 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:05.459248 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:05.459461 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:05.459465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:05.459726 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:05.459930 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:05.459934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.460707 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.460708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:05.461659 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:05.461662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.462096 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.462097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:05.462240 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.462764 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:05.462773 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:05.464286 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:05.466059 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:05.466065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:05.466066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:05.466293 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:05.466489 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:05.466490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:05.466490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:05.466590 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:05.466814 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:05.466819 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:05.466951 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 11:38:05.467033 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 11:38:05.467161 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260620 11:38:05.467932 9630 heartbeater.cc:347] Connected to a master server at 127.5.122.190:43755
I20260620 11:38:05.468063 9630 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:05.468261 9630 heartbeater.cc:511] Master 127.5.122.190:43755 requested a full tablet report, sending...
I20260620 11:38:05.468685 9171 ts_manager.cc:194] Registered new tserver with Master: f101711f035d4c5dbded91e10bb495d0 (127.5.122.131:44867)
I20260620 11:38:05.469074 9171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:39031
I20260620 11:38:05.470005 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:38:05.493352 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.485427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.5.122.129:40847)
0620 11:38:05.485602 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:05.485608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:05.485627 (+ 19us) server_negotiation.cc:407] Connection header received
0620 11:38:05.485702 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:05.485704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:05.485719 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:05.485789 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:05.486469 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.487202 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.487914 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.488250 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.488307 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.490586 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:05.490595 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:05.490595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:05.490602 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:05.492332 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.492646 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.492649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.492649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.492711 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.492880 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.492880 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.492880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.493032 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:05.493139 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:05.493204 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 11:38:05.493269 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:05.502657 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:05.494864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.5.122.190:43755)
0620 11:38:05.494932 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:05.494934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:05.494967 (+ 33us) server_negotiation.cc:407] Connection header received
0620 11:38:05.495046 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:05.495047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:05.495058 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:05.495103 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:05.495609 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.496117 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.496852 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:05.497061 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:05.497147 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:05.499581 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:05.499583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:05.499583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:05.499590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:05.501452 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.501808 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.501809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.501810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.501866 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:05.502055 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:05.502055 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:05.502056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:05.502217 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:05.502298 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:05.502448 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 11:38:05.502516 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 11:38:05.502899 9171 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47468
I20260620 11:38:05.503280 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 9233
I20260620 11:38:05.508469 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 9367
I20260620 11:38:05.512710 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 9501
I20260620 11:38:05.517141 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 9139
2026-06-20T11:38:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1883 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9642](info): set up 2 sockets
Jun 20 11:38:05 dist-test-slave-szdr krb5kdc[9642](info): commencing operation
krb5kdc: starting...
W20260620 11:38:06.550112 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:06 dist-test-slave-szdr krb5kdc[9642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:06.676484 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37379
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:37379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:06.764442 9655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:06.764663 9655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:06.764688 9655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:06.766227 9655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:06.766279 9655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:06.766294 9655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:06.766307 9655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:06.766319 9655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:06.768962 9655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:37379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:06.769318 9655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:06.769618 9655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:06.776417 9663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:06.776422 9660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:06.776417 9661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:06.776708 9655 server_base.cc:1034] running on GCE node
I20260620 11:38:06.777163 9655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:06.777539 9655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:06.778751 9655 hybrid_clock.cc:648] HybridClock initialized: now 1781955486778730 us; error 32 us; skew 500 ppm
Jun 20 11:38:06 dist-test-slave-szdr krb5kdc[9642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955486, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:06.782282 9655 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:06.782728 9655 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:06.782800 9655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:06.782953 9655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:06.784742 9655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6"
format_stamp: "Formatted at 2026-06-20 11:38:06 on dist-test-slave-szdr"
server_key: "9e69d39053ecc3c337d6f5826c70d1a9"
server_key_iv: "e7d7b6d047a40630762581da8984d76f"
server_key_version: "encryptionkey@0"
I20260620 11:38:06.785204 9655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6"
format_stamp: "Formatted at 2026-06-20 11:38:06 on dist-test-slave-szdr"
server_key: "9e69d39053ecc3c337d6f5826c70d1a9"
server_key_iv: "e7d7b6d047a40630762581da8984d76f"
server_key_version: "encryptionkey@0"
I20260620 11:38:06.787145 9655 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260620 11:38:06.788246 9669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:06.788434 9655 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:06.788544 9655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6"
format_stamp: "Formatted at 2026-06-20 11:38:06 on dist-test-slave-szdr"
server_key: "9e69d39053ecc3c337d6f5826c70d1a9"
server_key_iv: "e7d7b6d047a40630762581da8984d76f"
server_key_version: "encryptionkey@0"
I20260620 11:38:06.788625 9655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:06.818282 9655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:06.820313 9655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:06.820516 9655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:06.824862 9655 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:37379
I20260620 11:38:06.824916 9721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:37379 every 8 connection(s)
I20260620 11:38:06.825243 9655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:06.826347 9722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:06.828889 9722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6: Bootstrap starting.
I20260620 11:38:06.829603 9722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:06.830032 9722 log.cc:824] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:06.830842 9722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6: No bootstrap required, opened a new log
I20260620 11:38:06.832491 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 9655
I20260620 11:38:06.832625 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:06.832513 9722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37379 } }
I20260620 11:38:06.832731 9722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:06.832813 9722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27fa6fcb7ada4a7d8a13358ee891e2c6, State: Initialized, Role: FOLLOWER
I20260620 11:38:06.832875 5610 external_mini_cluster.cc:1428] Setting key b443f9ba79c6e9e91dfcdfa8465afb83
I20260620 11:38:06.832973 9722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37379 } }
I20260620 11:38:06.833060 9722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:06.833108 9722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:06.833178 9722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:06.833891 9722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37379 } }
I20260620 11:38:06.834066 9722 leader_election.cc:302] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27fa6fcb7ada4a7d8a13358ee891e2c6; no voters:
I20260620 11:38:06.834270 9722 leader_election.cc:288] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:06.834299 9727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:06.834481 9727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [term 1 LEADER]: Becoming Leader. State: Replica: 27fa6fcb7ada4a7d8a13358ee891e2c6, State: Running, Role: LEADER
I20260620 11:38:06.834547 9722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:06.834599 9727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37379 } }
I20260620 11:38:06.834931 9727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27fa6fcb7ada4a7d8a13358ee891e2c6. Latest consensus state: current_term: 1 leader_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37379 } } }
I20260620 11:38:06.835026 9727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:06.834939 9729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fa6fcb7ada4a7d8a13358ee891e2c6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37379 } } }
I20260620 11:38:06.835180 9729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6 [sys.catalog]: This master's current role is: LEADER
W20260620 11:38:06.836683 9742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:38:06.836791 9742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:38:06.836900 9733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:06.837070 9733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:06.838866 9733 catalog_manager.cc:1347] Generated new cluster ID: 47ef932ddee0442aa458b6af6c1cee7f
I20260620 11:38:06.838944 9733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:38:06 dist-test-slave-szdr krb5kdc[9642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:06.846076 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:06.833485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.5.122.190:37379)
0620 11:38:06.834172 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:06.834183 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:06.834228 (+ 45us) server_negotiation.cc:407] Connection header received
0620 11:38:06.834390 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:06.834399 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:06.834464 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:06.835569 (+ 1105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:06.835699 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:06.837241 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:06.838414 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:06.839073 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:06.839204 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:06.842123 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:06.842129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:06.842129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:06.842145 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:06.844238 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:06.844669 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:06.844673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:06.844673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:06.844797 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:06.845016 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:06.845017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:06.845017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:06.845380 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:06.845547 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:06.845653 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 11:38:06.845802 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":558,"threads_started":1}
I20260620 11:38:06.847247 9733 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:06.847741 9733 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:06.851251 9733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6: Generated new TSK 0
I20260620 11:38:06.851425 9733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:38:06.881737 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37379
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:06.991490 9748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:06.991787 9748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:06.991838 9748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:06.993325 9748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:06.993413 9748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:06.993466 9748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:06.995407 9748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:06.995729 9748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:06.995985 9748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:07.002282 9756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:07.002310 9753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:07.002300 9754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:07.002588 9748 server_base.cc:1034] running on GCE node
I20260620 11:38:07.002861 9748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:07.003197 9748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:07.004386 9748 hybrid_clock.cc:648] HybridClock initialized: now 1781955487004366 us; error 38 us; skew 500 ppm
Jun 20 11:38:07 dist-test-slave-szdr krb5kdc[9642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955487, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:07.007251 9748 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:38:07.007706 9748 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:07.007762 9748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:07.007918 9748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:07.009078 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "83ab11505b9f459b9755c756928cbf79"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "9981d9a577d43033317668e11546a4d6"
server_key_iv: "8b01efcf3a93c0046d9cadee466a77b5"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.009486 9748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "83ab11505b9f459b9755c756928cbf79"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "9981d9a577d43033317668e11546a4d6"
server_key_iv: "8b01efcf3a93c0046d9cadee466a77b5"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.011080 9748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:07.011929 9762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.012145 9748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:38:07.012247 9748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "83ab11505b9f459b9755c756928cbf79"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "9981d9a577d43033317668e11546a4d6"
server_key_iv: "8b01efcf3a93c0046d9cadee466a77b5"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.012313 9748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:07.022658 9748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:07.025209 9748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:07.025334 9748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:07.025540 9748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:07.025892 9748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:07.025929 9748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.025952 9748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:07.026028 9748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.032269 9748 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:46597
I20260620 11:38:07.032315 9875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:46597 every 8 connection(s)
I20260620 11:38:07.032648 9748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:07.038233 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 9748
I20260620 11:38:07.038353 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:07.038538 5610 external_mini_cluster.cc:1428] Setting key b3abf38f5dfe1a191b5c42cb3f6c8efc
Jun 20 11:38:07 dist-test-slave-szdr krb5kdc[9642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955487, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:07.044551 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.033587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:36417 (local address 127.5.122.190:37379)
0620 11:38:07.033664 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.033665 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.033902 (+ 237us) server_negotiation.cc:407] Connection header received
0620 11:38:07.034498 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.034499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.034509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.034563 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:07.035886 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.036684 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.038316 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.038547 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.038648 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.041247 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:07.041249 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:07.041249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:07.041262 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:07.042952 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.043442 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.043444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.043445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.043513 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.043825 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.043826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.043827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.043960 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:07.044040 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.044364 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.044454 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 11:38:07.044771 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.033568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37379 (local address 127.5.122.129:36417)
0620 11:38:07.033761 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.033807 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.034399 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.034660 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.034664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.034958 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:07.035748 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.035757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.036798 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.036800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.038209 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.038213 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.038662 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.038663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.038841 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.039485 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:07.039496 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:07.041119 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:07.043068 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:07.043078 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:07.043082 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:07.043351 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:07.043593 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:07.043594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:07.043594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:07.043727 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:07.044042 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:07.044050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:07.044210 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.044328 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.044509 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260620 11:38:07.045261 9876 heartbeater.cc:347] Connected to a master server at 127.5.122.190:37379
I20260620 11:38:07.045368 9876 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:07.045548 9876 heartbeater.cc:511] Master 127.5.122.190:37379 requested a full tablet report, sending...
I20260620 11:38:07.046149 9686 ts_manager.cc:194] Registered new tserver with Master: 83ab11505b9f459b9755c756928cbf79 (127.5.122.129:46597)
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
I20260620 11:38:07.046859 9686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:36417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:38:07.065353 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37379
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:07.150960 9881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:07.151230 9881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:07.151257 9881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:07.152442 9881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:07.152482 9881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:07.152510 9881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:07.154038 9881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:07.154240 9881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:07.154424 9881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:07.159840 9889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:07.159840 9887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:07.159905 9881 server_base.cc:1034] running on GCE node
W20260620 11:38:07.160030 9886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:07.160308 9881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:07.160575 9881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:07.161798 9881 hybrid_clock.cc:648] HybridClock initialized: now 1781955487161792 us; error 44 us; skew 500 ppm
Jun 20 11:38:07 dist-test-slave-szdr krb5kdc[9642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955487, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:07.164521 9881 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:38:07.164824 9881 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:07.164870 9881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:07.164968 9881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:07.166112 9881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "0fd51ec8339345489d5ac1aacd4f251f"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "67d41ed8adf4ec6591cf066b3ba45b24"
server_key_iv: "17aea9c020c2dc16d1c9911aa1c2a91e"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.166460 9881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "0fd51ec8339345489d5ac1aacd4f251f"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "67d41ed8adf4ec6591cf066b3ba45b24"
server_key_iv: "17aea9c020c2dc16d1c9911aa1c2a91e"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.167775 9881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:07.168556 9895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.168798 9881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:07.168898 9881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "0fd51ec8339345489d5ac1aacd4f251f"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "67d41ed8adf4ec6591cf066b3ba45b24"
server_key_iv: "17aea9c020c2dc16d1c9911aa1c2a91e"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.168960 9881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:07.193807 9881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:07.196082 9881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:07.196208 9881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:07.196470 9881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:07.196854 9881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:07.196890 9881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.196915 9881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:07.196961 9881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.203173 9881 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:41039
I20260620 11:38:07.203181 10008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:41039 every 8 connection(s)
I20260620 11:38:07.203555 9881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
Jun 20 11:38:07 dist-test-slave-szdr krb5kdc[9642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955487, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:07.211220 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 9881
I20260620 11:38:07.211356 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:07.211568 5610 external_mini_cluster.cc:1428] Setting key 4dfe34f287dec64fbbe52c41118e710e
I20260620 11:38:07.213785 10011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.204348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37379 (local address 127.5.122.130:60099)
0620 11:38:07.204531 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.204571 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.205056 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.205305 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.205309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.205583 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:07.206282 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.206289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.206983 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.206983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.207835 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.207838 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.208204 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.208205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.208330 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.208827 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:07.208836 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:07.210378 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:07.212368 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:07.212376 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:07.212379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:07.212643 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:07.212868 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:07.212868 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:07.212869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:07.212975 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:07.213272 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:07.213275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:07.213395 (+ 120us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.213484 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.213612 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260620 11:38:07.213788 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.204368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:60099 (local address 127.5.122.190:37379)
0620 11:38:07.204435 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.204437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.204650 (+ 213us) server_negotiation.cc:407] Connection header received
0620 11:38:07.205155 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.205156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.205168 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.205224 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:07.206378 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.206895 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.207925 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.208124 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.208178 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.210481 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:07.210483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:07.210483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:07.210491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:07.212141 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.212731 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.212732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.212733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.212794 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.213048 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.213049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.213049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.213206 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:07.213275 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.213537 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.213642 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:38:07.214506 10009 heartbeater.cc:347] Connected to a master server at 127.5.122.190:37379
I20260620 11:38:07.214632 10009 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:07.214819 10009 heartbeater.cc:511] Master 127.5.122.190:37379 requested a full tablet report, sending...
I20260620 11:38:07.215225 9686 ts_manager.cc:194] Registered new tserver with Master: 0fd51ec8339345489d5ac1aacd4f251f (127.5.122.130:41039)
I20260620 11:38:07.215817 9686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:60099
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:38:07.239235 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37379
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:07.323554 10014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:07.323793 10014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:07.323819 10014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:07.324988 10014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:07.325032 10014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:07.325059 10014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:07.326555 10014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:37379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:07.326766 10014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:07.326952 10014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:07.332044 10019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:07.332044 10022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:07.332152 10020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:07.332314 10014 server_base.cc:1034] running on GCE node
I20260620 11:38:07.332521 10014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:07.332796 10014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:07.334064 10014 hybrid_clock.cc:648] HybridClock initialized: now 1781955487334063 us; error 46 us; skew 500 ppm
Jun 20 11:38:07 dist-test-slave-szdr krb5kdc[9642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955487, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:07.337074 10014 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:38:07.337495 10014 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:07.337584 10014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:07.337743 10014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:07.338933 10014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "33922389d528406691dd1ad709baeba5"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "274614be29ba687bf6fbaf933a7f3f03"
server_key_iv: "2a7b9c5e60592506ca1641ae9d91cf38"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.339323 10014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "33922389d528406691dd1ad709baeba5"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "274614be29ba687bf6fbaf933a7f3f03"
server_key_iv: "2a7b9c5e60592506ca1641ae9d91cf38"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.340689 10014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:07.341509 10028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.341687 10014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:07.341796 10014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "33922389d528406691dd1ad709baeba5"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "274614be29ba687bf6fbaf933a7f3f03"
server_key_iv: "2a7b9c5e60592506ca1641ae9d91cf38"
server_key_version: "encryptionkey@0"
I20260620 11:38:07.341885 10014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:07.359867 10014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:07.361917 10014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:07.362136 10014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:07.362411 10014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:07.362767 10014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:07.362829 10014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.362871 10014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:07.362900 10014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:07.369031 10014 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:41193
I20260620 11:38:07.369069 10141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:41193 every 8 connection(s)
I20260620 11:38:07.369369 10014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:07.374607 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10014
I20260620 11:38:07.374712 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:07.374923 5610 external_mini_cluster.cc:1428] Setting key 0d6c3e9403904251dcd185b910551529
Jun 20 11:38:07 dist-test-slave-szdr krb5kdc[9642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955487, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:07.379302 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.370207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:56473 (local address 127.5.122.190:37379)
0620 11:38:07.370284 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.370286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.370524 (+ 238us) server_negotiation.cc:407] Connection header received
0620 11:38:07.371014 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.371016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.371027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.371083 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:07.372098 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.372715 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.373764 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.373954 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.374078 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.376688 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:07.376690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:07.376690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:07.376697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:07.377900 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.378363 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.378364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.378365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.378431 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.378742 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.378742 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.378743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.378852 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:07.378903 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.379168 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.379238 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 11:38:07.379571 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.370210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37379 (local address 127.5.122.131:56473)
0620 11:38:07.370389 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.370428 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.370927 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.371163 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.371180 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.371454 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:07.372010 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.372016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.372820 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.372821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.373680 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.373682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.374105 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.374106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.374213 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.374837 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:07.374846 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:07.376596 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:07.378046 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:07.378052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:07.378055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:07.378284 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:07.378532 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:07.378533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:07.378533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:07.378662 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:07.378916 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:07.378920 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:07.379044 (+ 124us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.379134 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.379274 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260620 11:38:07.380299 10142 heartbeater.cc:347] Connected to a master server at 127.5.122.190:37379
I20260620 11:38:07.380434 10142 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:07.380618 10142 heartbeater.cc:511] Master 127.5.122.190:37379 requested a full tablet report, sending...
I20260620 11:38:07.380987 9686 ts_manager.cc:194] Registered new tserver with Master: 33922389d528406691dd1ad709baeba5 (127.5.122.131:41193)
I20260620 11:38:07.381374 9686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:56473
I20260620 11:38:07.387178 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:38:07.395234 9726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.388536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.5.122.190:37379)
0620 11:38:07.388633 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.388636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.388719 (+ 83us) server_negotiation.cc:407] Connection header received
0620 11:38:07.388808 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.388809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.388820 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.388870 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:07.389459 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.390033 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.390790 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.391022 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.391108 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.392401 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:07.392402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:07.392403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:07.392411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:07.393943 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.394369 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.394371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.394371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.394435 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:07.394654 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:07.394655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:07.394655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:07.394819 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:07.394955 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.395042 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.395129 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 11:38:07.397061 9686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:07.399058 9686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:07.406651 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.401878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.5.122.129:46597)
0620 11:38:07.402262 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.402267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.402278 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:38:07.402349 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.402351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.402370 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.402488 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:07.402738 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.403904 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.405588 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.406314 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.406379 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.406429 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.406528 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.406565 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":93,"threads_started":1}
I20260620 11:38:07.406643 10153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.401719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46597 (local address 127.0.0.1:41104)
0620 11:38:07.401834 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.401879 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.401941 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.402561 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.402564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.402566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:07.402650 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.402653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.404025 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.404027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.405485 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.405488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.406382 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.406383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.406460 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.406464 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.406517 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.406561 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":83,"thread_start_us":67,"threads_started":1}
I20260620 11:38:07.406733 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.402211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.5.122.130:41039)
0620 11:38:07.402413 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.402416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.402425 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:07.402459 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.402461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.402477 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.402697 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:07.402966 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.404024 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.405598 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.406190 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.406285 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.406346 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.406607 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.406651 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":115,"threads_started":1}
I20260620 11:38:07.406868 10157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.402085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.5.122.131:41193)
0620 11:38:07.402533 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.402538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.402547 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:07.402584 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.402586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.402602 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.402677 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:07.402941 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.403880 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.405362 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.406159 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.406243 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.406318 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.406609 (+ 291us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.406657 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":79,"threads_started":1}
I20260620 11:38:07.407085 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.401812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:41193 (local address 127.0.0.1:41314)
0620 11:38:07.402070 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.402082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.402140 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.402772 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.402772 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.402775 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:07.402819 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.402821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.404013 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.404014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.405255 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.405258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.406257 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.406259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.406342 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.406357 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.406428 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.406491 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":93,"thread_start_us":101,"threads_started":1}
I20260620 11:38:07.407086 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.401949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41039 (local address 127.0.0.1:38874)
0620 11:38:07.402218 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.402230 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.402280 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.402795 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.402796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.402799 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:07.402851 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.402880 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.404123 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.404124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.405511 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.405513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.406376 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.406377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.406447 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.406450 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.406495 (+ 45us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.406523 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":170,"thread_start_us":99,"threads_started":1}
I20260620 11:38:07.407792 9810 tablet_service.cc:1467] Processing CreateTablet for tablet 2b701319ab4e41e686880913d3e8fc66 (DEFAULT_TABLE table=test-table [id=ae38d480d67145759f456c2904c7d4f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:07.407801 9943 tablet_service.cc:1467] Processing CreateTablet for tablet 2b701319ab4e41e686880913d3e8fc66 (DEFAULT_TABLE table=test-table [id=ae38d480d67145759f456c2904c7d4f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:07.407801 10076 tablet_service.cc:1467] Processing CreateTablet for tablet 2b701319ab4e41e686880913d3e8fc66 (DEFAULT_TABLE table=test-table [id=ae38d480d67145759f456c2904c7d4f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:07.408154 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b701319ab4e41e686880913d3e8fc66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:07.408150 9943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b701319ab4e41e686880913d3e8fc66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:07.408150 10076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b701319ab4e41e686880913d3e8fc66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:07.411018 10159 tablet_bootstrap.cc:492] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: Bootstrap starting.
I20260620 11:38:07.411579 10161 tablet_bootstrap.cc:492] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Bootstrap starting.
I20260620 11:38:07.411899 10159 tablet_bootstrap.cc:654] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:07.412249 10161 tablet_bootstrap.cc:654] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:07.412581 10159 log.cc:824] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:07.412863 10161 log.cc:824] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:07.413232 10160 tablet_bootstrap.cc:492] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Bootstrap starting.
I20260620 11:38:07.413519 10159 tablet_bootstrap.cc:492] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: No bootstrap required, opened a new log
I20260620 11:38:07.413600 10159 ts_tablet_manager.cc:1397] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:38:07.413868 10160 tablet_bootstrap.cc:654] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:07.414489 10160 log.cc:824] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:07.414542 10161 tablet_bootstrap.cc:492] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: No bootstrap required, opened a new log
I20260620 11:38:07.414637 10161 ts_tablet_manager.cc:1397] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:38:07.415285 10160 tablet_bootstrap.cc:492] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: No bootstrap required, opened a new log
I20260620 11:38:07.415357 10160 ts_tablet_manager.cc:1397] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:38:07.415832 10159 raft_consensus.cc:348] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.416025 10159 raft_consensus.cc:374] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:07.416069 10159 raft_consensus.cc:729] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ab11505b9f459b9755c756928cbf79, State: Initialized, Role: FOLLOWER
I20260620 11:38:07.416180 10159 consensus_queue.cc:260] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.416479 9876 heartbeater.cc:503] Master 127.5.122.190:37379 was elected leader, sending a full tablet report...
I20260620 11:38:07.416488 10159 ts_tablet_manager.cc:1428] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 11:38:07.416767 10161 raft_consensus.cc:348] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.416944 10161 raft_consensus.cc:374] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:07.416981 10161 raft_consensus.cc:729] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd51ec8339345489d5ac1aacd4f251f, State: Initialized, Role: FOLLOWER
I20260620 11:38:07.417100 10161 consensus_queue.cc:260] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.417384 10161 ts_tablet_manager.cc:1428] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:38:07.417439 10009 heartbeater.cc:503] Master 127.5.122.190:37379 was elected leader, sending a full tablet report...
I20260620 11:38:07.417469 10160 raft_consensus.cc:348] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.417611 10160 raft_consensus.cc:374] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:07.417645 10160 raft_consensus.cc:729] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33922389d528406691dd1ad709baeba5, State: Initialized, Role: FOLLOWER
I20260620 11:38:07.417747 10160 consensus_queue.cc:260] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.418071 10160 ts_tablet_manager.cc:1428] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:38:07.418107 10142 heartbeater.cc:503] Master 127.5.122.190:37379 was elected leader, sending a full tablet report...
W20260620 11:38:07.454165 10010 tablet.cc:2369] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:38:07.533468 9877 tablet.cc:2369] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:38:07.620028 10143 tablet.cc:2369] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:38:07.792454 10165 raft_consensus.cc:482] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:07.792588 10165 raft_consensus.cc:504] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.793018 10165 leader_election.cc:288] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fd51ec8339345489d5ac1aacd4f251f (127.5.122.130:41039), 33922389d528406691dd1ad709baeba5 (127.5.122.131:41193)
I20260620 11:38:07.796293 10169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.793213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41039 (local address 127.5.122.129:34545)
0620 11:38:07.793358 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.793371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.793480 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.793728 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.793729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.793731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:07.793823 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.793824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.794539 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.794541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.795381 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.795384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.795970 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.795971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.796057 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.796061 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.796154 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.796199 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":19,"thread_start_us":65,"threads_started":1}
I20260620 11:38:07.796285 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.793304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:34545 (local address 127.5.122.130:41039)
0620 11:38:07.793393 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.793395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.793443 (+ 48us) server_negotiation.cc:407] Connection header received
0620 11:38:07.793601 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.793603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.793631 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.793721 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:07.793905 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.794439 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.795492 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.795865 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.795969 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.795979 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.796164 (+ 185us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.796218 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 11:38:07.796288 10157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.793323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:57383 (local address 127.5.122.131:41193)
0620 11:38:07.793372 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.793375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.793427 (+ 52us) server_negotiation.cc:407] Connection header received
0620 11:38:07.793515 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.793516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.793526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.793580 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:07.793757 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.794292 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.795493 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.795893 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.795994 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.796002 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.796180 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.796216 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 11:38:07.796319 10168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.793209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:41193 (local address 127.5.122.129:57383)
0620 11:38:07.793347 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.793360 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:07.793434 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:07.793584 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:07.793585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:07.793587 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:07.793672 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.793673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.794513 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.794514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.795382 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:07.795385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.795995 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.795996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:07.796081 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.796085 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:07.796171 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 11:38:07.796215 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":28,"thread_start_us":85,"threads_started":1}
I20260620 11:38:07.796672 9963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b701319ab4e41e686880913d3e8fc66" candidate_uuid: "83ab11505b9f459b9755c756928cbf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd51ec8339345489d5ac1aacd4f251f" is_pre_election: true
I20260620 11:38:07.796811 9963 raft_consensus.cc:2454] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ab11505b9f459b9755c756928cbf79 in term 0.
I20260620 11:38:07.796790 10096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b701319ab4e41e686880913d3e8fc66" candidate_uuid: "83ab11505b9f459b9755c756928cbf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33922389d528406691dd1ad709baeba5" is_pre_election: true
I20260620 11:38:07.796952 10096 raft_consensus.cc:2454] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83ab11505b9f459b9755c756928cbf79 in term 0.
I20260620 11:38:07.797099 9766 leader_election.cc:302] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd51ec8339345489d5ac1aacd4f251f, 83ab11505b9f459b9755c756928cbf79; no voters:
I20260620 11:38:07.797251 10165 raft_consensus.cc:2781] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:07.797312 10165 raft_consensus.cc:482] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:07.797335 10165 raft_consensus.cc:3037] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:07.798095 10165 raft_consensus.cc:504] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.798255 10165 leader_election.cc:288] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [CANDIDATE]: Term 1 election: Requested vote from peers 0fd51ec8339345489d5ac1aacd4f251f (127.5.122.130:41039), 33922389d528406691dd1ad709baeba5 (127.5.122.131:41193)
I20260620 11:38:07.798437 9963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b701319ab4e41e686880913d3e8fc66" candidate_uuid: "83ab11505b9f459b9755c756928cbf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fd51ec8339345489d5ac1aacd4f251f"
I20260620 11:38:07.798535 9963 raft_consensus.cc:3037] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:07.798550 10096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b701319ab4e41e686880913d3e8fc66" candidate_uuid: "83ab11505b9f459b9755c756928cbf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33922389d528406691dd1ad709baeba5"
I20260620 11:38:07.798655 10096 raft_consensus.cc:3037] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:07.799439 9963 raft_consensus.cc:2454] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ab11505b9f459b9755c756928cbf79 in term 1.
I20260620 11:38:07.799649 9766 leader_election.cc:302] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fd51ec8339345489d5ac1aacd4f251f, 83ab11505b9f459b9755c756928cbf79; no voters:
I20260620 11:38:07.799695 10096 raft_consensus.cc:2454] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83ab11505b9f459b9755c756928cbf79 in term 1.
I20260620 11:38:07.799769 10165 raft_consensus.cc:2781] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:07.799935 10165 raft_consensus.cc:686] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 1 LEADER]: Becoming Leader. State: Replica: 83ab11505b9f459b9755c756928cbf79, State: Running, Role: LEADER
I20260620 11:38:07.800051 10165 consensus_queue.cc:237] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } }
I20260620 11:38:07.800968 9685 catalog_manager.cc:5495] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83ab11505b9f459b9755c756928cbf79 (127.5.122.129). New cstate: current_term: 1 leader_uuid: "83ab11505b9f459b9755c756928cbf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83ab11505b9f459b9755c756928cbf79" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 } health_report { overall_health: UNKNOWN } } }
I20260620 11:38:07.848613 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.846027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.5.122.129:46597)
0620 11:38:07.846096 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:07.846098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:07.846107 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:07.846158 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:07.846158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:07.846181 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:07.846277 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:07.846486 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.846959 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.847749 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:07.847954 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:07.848035 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:07.848230 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:07.848342 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:07.848487 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:07.848505 (+ 18us) server_negotiation.cc:299] Negotiation successful
0620 11:38:07.848527 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:38:07.850564 9963 raft_consensus.cc:1261] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 1 FOLLOWER]: Refusing update from remote peer 83ab11505b9f459b9755c756928cbf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:07.850579 10096 raft_consensus.cc:1261] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 1 FOLLOWER]: Refusing update from remote peer 83ab11505b9f459b9755c756928cbf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:07.850961 10170 consensus_queue.cc:1035] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33922389d528406691dd1ad709baeba5" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 41193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:07.851105 10165 consensus_queue.cc:1035] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fd51ec8339345489d5ac1aacd4f251f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:07.853513 10173 mvcc.cc:205] Tried to move back new op lower bound from 7298889678232068096 to 7298889678029012992. Current Snapshot: MvccSnapshot[applied={T|T < 7298889678232068096}]
I20260620 11:38:07.854692 10175 mvcc.cc:205] Tried to move back new op lower bound from 7298889678232068096 to 7298889678029012992. Current Snapshot: MvccSnapshot[applied={T|T < 7298889678232068096}]
I20260620 11:38:07.856362 9685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:07.856482 9685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:07.857300 9685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27fa6fcb7ada4a7d8a13358ee891e2c6: Sending DeleteTablet for 3 replicas of tablet 2b701319ab4e41e686880913d3e8fc66
I20260620 11:38:07.857698 9810 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b701319ab4e41e686880913d3e8fc66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:07 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:41104
I20260620 11:38:07.857738 9943 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b701319ab4e41e686880913d3e8fc66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:07 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:38874
I20260620 11:38:07.857738 10076 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b701319ab4e41e686880913d3e8fc66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:07 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:41314
I20260620 11:38:07.857939 10183 tablet_replica.cc:330] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79: stopping tablet replica
I20260620 11:38:07.858072 10183 raft_consensus.cc:2229] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:38:07.858174 10183 raft_consensus.cc:2258] T 2b701319ab4e41e686880913d3e8fc66 P 83ab11505b9f459b9755c756928cbf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:07.858225 10184 tablet_replica.cc:330] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: stopping tablet replica
I20260620 11:38:07.858296 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 9748
I20260620 11:38:07.858333 10184 raft_consensus.cc:2229] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:07.858865 10185 tablet_replica.cc:330] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: stopping tablet replica
I20260620 11:38:07.859010 10185 raft_consensus.cc:2229] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:07.859148 10185 raft_consensus.cc:2258] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:07.859323 10184 raft_consensus.cc:2258] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:07.859726 10185 ts_tablet_manager.cc:1905] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:07.859844 10184 ts_tablet_manager.cc:1905] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:07.862112 10184 ts_tablet_manager.cc:1918] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:07.862114 10185 ts_tablet_manager.cc:1918] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:07.862216 10184 log.cc:1196] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/2b701319ab4e41e686880913d3e8fc66
I20260620 11:38:07.862216 10185 log.cc:1196] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/2b701319ab4e41e686880913d3e8fc66
I20260620 11:38:07.862509 10184 ts_tablet_manager.cc:1939] T 2b701319ab4e41e686880913d3e8fc66 P 33922389d528406691dd1ad709baeba5: Deleting consensus metadata
I20260620 11:38:07.862504 10185 ts_tablet_manager.cc:1939] T 2b701319ab4e41e686880913d3e8fc66 P 0fd51ec8339345489d5ac1aacd4f251f: Deleting consensus metadata
I20260620 11:38:07.862988 9670 catalog_manager.cc:4841] TS 33922389d528406691dd1ad709baeba5 (127.5.122.131:41193): tablet 2b701319ab4e41e686880913d3e8fc66 (table test-table [id=ae38d480d67145759f456c2904c7d4f2]) successfully deleted
I20260620 11:38:07.863082 9673 catalog_manager.cc:4841] TS 0fd51ec8339345489d5ac1aacd4f251f (127.5.122.130:41039): tablet 2b701319ab4e41e686880913d3e8fc66 (table test-table [id=ae38d480d67145759f456c2904c7d4f2]) successfully deleted
W20260620 11:38:07.865283 9673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:46597 (error 108)
I20260620 11:38:07.865538 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 9881
I20260620 11:38:07.865868 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:07.865743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46597 (local address 127.0.0.1:41114)
0620 11:38:07.865795 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:07.865827 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 11:38:07.866009 9673 catalog_manager.cc:4584] TS 83ab11505b9f459b9755c756928cbf79 (127.5.122.129:46597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b701319ab4e41e686880913d3e8fc66: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46597: connect: Connection refused (error 111)
I20260620 11:38:07.870100 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10014
I20260620 11:38:07.874877 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 9655
2026-06-20T11:38:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2358 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T11:38:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:07Z Disabled control of system clock
I20260620 11:38:07.904121 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:33619
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:33619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 11:38:07.988523 10190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:07.988763 10190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:07.988786 10190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:07.989953 10190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:07.990023 10190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:07.990036 10190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:07.990049 10190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:07.990061 10190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:07.991400 10190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:33619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:33619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:07.991644 10190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:07.991904 10190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:07.997006 10198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:07.997071 10196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:07.997182 10195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:07.997090 10190 server_base.cc:1034] running on GCE node
I20260620 11:38:07.997617 10190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:07.997943 10190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:07.999161 10190 hybrid_clock.cc:648] HybridClock initialized: now 1781955487999119 us; error 57 us; skew 500 ppm
I20260620 11:38:08.000525 10190 webserver.cc:460] Webserver started at http://127.5.122.190:44885/ using document root <none> and password file <none>
I20260620 11:38:08.000814 10190 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:08.000893 10190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:08.001044 10190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:08.002297 10190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "100cd3eb32e1206665f188c6c3b24c9e"
server_key_iv: "84e0417e4e3656c86f5e97d93e49a3b4"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.002696 10190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "100cd3eb32e1206665f188c6c3b24c9e"
server_key_iv: "84e0417e4e3656c86f5e97d93e49a3b4"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.004093 10190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 11:38:08.004877 10204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.005060 10190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:08.005168 10190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb"
format_stamp: "Formatted at 2026-06-20 11:38:07 on dist-test-slave-szdr"
server_key: "100cd3eb32e1206665f188c6c3b24c9e"
server_key_iv: "84e0417e4e3656c86f5e97d93e49a3b4"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.005259 10190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:08.008917 10190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:08.009208 10190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:08.009364 10190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:08.013707 10190 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:33619
I20260620 11:38:08.013756 10256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:33619 every 8 connection(s)
I20260620 11:38:08.014191 10190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:08.014899 10257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:08.017618 10257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb: Bootstrap starting.
I20260620 11:38:08.018252 10257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:08.018486 10257 log.cc:824] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:08.018823 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10190
I20260620 11:38:08.018944 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:08.019124 5610 external_mini_cluster.cc:1428] Setting key 3a26f9c118cb0a4c4fdba2ece99866b4
I20260620 11:38:08.019342 10257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb: No bootstrap required, opened a new log
I20260620 11:38:08.021770 10257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 33619 } }
I20260620 11:38:08.021986 10257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:08.022045 10257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90eed14f51b54cbb8f6f0f2c590fe2bb, State: Initialized, Role: FOLLOWER
I20260620 11:38:08.022190 10257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 33619 } }
I20260620 11:38:08.022285 10257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:08.022342 10257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:08.022389 10257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:08.022426 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.019704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.5.122.190:33619)
0620 11:38:08.019952 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.019961 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.020013 (+ 52us) server_negotiation.cc:407] Connection header received
0620 11:38:08.020812 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.020822 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.020886 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.021061 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.021783 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.021787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.021787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.021793 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.021842 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.021856 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.021933 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.022135 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.022196 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
I20260620 11:38:08.023087 10257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 33619 } }
I20260620 11:38:08.023232 10257 leader_election.cc:302] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90eed14f51b54cbb8f6f0f2c590fe2bb; no voters:
I20260620 11:38:08.023429 10257 leader_election.cc:288] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:08.023455 10262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:08.023571 10262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [term 1 LEADER]: Becoming Leader. State: Replica: 90eed14f51b54cbb8f6f0f2c590fe2bb, State: Running, Role: LEADER
I20260620 11:38:08.023660 10262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 33619 } }
I20260620 11:38:08.023743 10257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:08.024005 10263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 33619 } } }
I20260620 11:38:08.024118 10263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:08.024005 10264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90eed14f51b54cbb8f6f0f2c590fe2bb. Latest consensus state: current_term: 1 leader_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90eed14f51b54cbb8f6f0f2c590fe2bb" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 33619 } } }
I20260620 11:38:08.024266 10264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:08.024353 10267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:08.024915 10267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:08.026672 10267 catalog_manager.cc:1347] Generated new cluster ID: e0ee1f633f37414faf9f389484f4803e
I20260620 11:38:08.026732 10267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:08.044173 10267 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:08.044384 10267 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:08.046293 10267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb: Generated new TSK 0
I20260620 11:38:08.046497 10267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:38:08.050459 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:33619
--builtin_ntp_servers=127.5.122.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 11:38:08.138315 10281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:08.138594 10281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:08.138633 10281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:08.139906 10281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:08.139995 10281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:08.140041 10281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:08.141489 10281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:33619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:08.141803 10281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:08.142163 10281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:08.148474 10289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:08.148602 10281 server_base.cc:1034] running on GCE node
W20260620 11:38:08.148469 10286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.148464 10287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:08.148885 10281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:08.149264 10281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:08.150453 10281 hybrid_clock.cc:648] HybridClock initialized: now 1781955488150438 us; error 28 us; skew 500 ppm
I20260620 11:38:08.151976 10281 webserver.cc:460] Webserver started at http://127.5.122.129:41869/ using document root <none> and password file <none>
I20260620 11:38:08.152243 10281 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:08.152300 10281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:08.152423 10281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:08.153815 10281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "44c072588eb444b7b695fd44a1bd46d7"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "8e1f3b6627b2c36534a2fadc9967ed80"
server_key_iv: "4deb825801db48a82c7698f8d881e400"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.154232 10281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "44c072588eb444b7b695fd44a1bd46d7"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "8e1f3b6627b2c36534a2fadc9967ed80"
server_key_iv: "4deb825801db48a82c7698f8d881e400"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.155922 10281 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 11:38:08.156859 10295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.157187 10281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:08.157323 10281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "44c072588eb444b7b695fd44a1bd46d7"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "8e1f3b6627b2c36534a2fadc9967ed80"
server_key_iv: "4deb825801db48a82c7698f8d881e400"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.157433 10281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:08.161329 10281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:08.161672 10281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:08.161849 10281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:08.162124 10281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:08.162508 10281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:08.162549 10281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.162606 10281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:08.162634 10281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.169255 10281 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:43911
I20260620 11:38:08.169344 10408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:43911 every 8 connection(s)
I20260620 11:38:08.169734 10281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:08.173053 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.170627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:34019 (local address 127.5.122.190:33619)
0620 11:38:08.170744 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.170746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.170977 (+ 231us) server_negotiation.cc:407] Connection header received
0620 11:38:08.171467 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.171468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.171473 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.171548 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.172800 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.172802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.172802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.172807 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.172823 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.172827 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.172884 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.172979 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.172992 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260620 11:38:08.173269 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.170610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:33619 (local address 127.5.122.129:34019)
0620 11:38:08.170828 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.170872 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.171369 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.171628 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.171631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.171640 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.172626 (+ 986us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.172626 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.172658 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.172691 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.172899 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.172904 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.172906 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.172968 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.173024 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260620 11:38:08.173878 10409 heartbeater.cc:347] Connected to a master server at 127.5.122.190:33619
I20260620 11:38:08.173997 10409 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:08.174167 10409 heartbeater.cc:511] Master 127.5.122.190:33619 requested a full tablet report, sending...
I20260620 11:38:08.174723 10221 ts_manager.cc:194] Registered new tserver with Master: 44c072588eb444b7b695fd44a1bd46d7 (127.5.122.129:43911)
I20260620 11:38:08.177455 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10281
I20260620 11:38:08.177590 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:08.177805 5610 external_mini_cluster.cc:1428] Setting key a435114c0d98e94f1e88d0f6b34dc7aa
I20260620 11:38:08.178829 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:33619
--builtin_ntp_servers=127.5.122.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 11:38:08.265254 10412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:08.265482 10412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:08.265507 10412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:08.266862 10412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:08.266913 10412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:08.266944 10412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:08.268404 10412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:33619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:08.268707 10412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:08.269052 10412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:08.274427 10418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.274436 10417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:08.274801 10412 server_base.cc:1034] running on GCE node
W20260620 11:38:08.274459 10420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:08.275095 10412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:08.275362 10412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:08.276511 10412 hybrid_clock.cc:648] HybridClock initialized: now 1781955488276501 us; error 39 us; skew 500 ppm
I20260620 11:38:08.277952 10412 webserver.cc:460] Webserver started at http://127.5.122.130:32999/ using document root <none> and password file <none>
I20260620 11:38:08.278275 10412 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:08.278353 10412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:08.278509 10412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:08.279624 10412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "7ef603bb2410488e8b300eaf75695fd7"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "653862b1015d0688c7783a8a866172bc"
server_key_iv: "cd1d7123f278af15ce35b6c6d8fa40b0"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.280014 10412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "7ef603bb2410488e8b300eaf75695fd7"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "653862b1015d0688c7783a8a866172bc"
server_key_iv: "cd1d7123f278af15ce35b6c6d8fa40b0"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.281432 10412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:08.282356 10426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.282644 10412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:08.282768 10412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "7ef603bb2410488e8b300eaf75695fd7"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "653862b1015d0688c7783a8a866172bc"
server_key_iv: "cd1d7123f278af15ce35b6c6d8fa40b0"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.282845 10412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:08.286513 10412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:08.286818 10412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:08.286969 10412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:08.287204 10412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:08.287537 10412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:08.287598 10412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.287642 10412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:08.287668 10412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.293426 10412 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:33727
I20260620 11:38:08.293507 10539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:33727 every 8 connection(s)
I20260620 11:38:08.293856 10412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:08.296896 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.294690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:33723 (local address 127.5.122.190:33619)
0620 11:38:08.294790 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.294792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.295008 (+ 216us) server_negotiation.cc:407] Connection header received
0620 11:38:08.295478 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.295480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.295485 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.295540 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.296691 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.296692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.296693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.296697 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.296713 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.296718 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.296772 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.296827 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.296839 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 11:38:08.297127 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.294649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:33619 (local address 127.5.122.130:33723)
0620 11:38:08.294877 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.294913 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.295389 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.295595 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.295599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.295607 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.296537 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.296537 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.296567 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.296597 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.296770 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.296772 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.296774 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.296834 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.296882 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260620 11:38:08.297693 10540 heartbeater.cc:347] Connected to a master server at 127.5.122.190:33619
I20260620 11:38:08.297818 10540 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:08.298019 10540 heartbeater.cc:511] Master 127.5.122.190:33619 requested a full tablet report, sending...
I20260620 11:38:08.298444 10221 ts_manager.cc:194] Registered new tserver with Master: 7ef603bb2410488e8b300eaf75695fd7 (127.5.122.130:33727)
I20260620 11:38:08.304100 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10412
I20260620 11:38:08.304232 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:08.304409 5610 external_mini_cluster.cc:1428] Setting key 4f12489b2b772ca2ed5210a0ac4b5896
I20260620 11:38:08.305486 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:33619
--builtin_ntp_servers=127.5.122.148:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 11:38:08.391824 10543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:08.392055 10543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:08.392078 10543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:08.393253 10543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:08.393291 10543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:08.393318 10543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:08.394829 10543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:33619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:08.395043 10543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:08.395303 10543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:08.400605 10551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.400662 10548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.400605 10549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:08.400942 10543 server_base.cc:1034] running on GCE node
I20260620 11:38:08.401182 10543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:08.401410 10543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:08.402552 10543 hybrid_clock.cc:648] HybridClock initialized: now 1781955488402538 us; error 30 us; skew 500 ppm
I20260620 11:38:08.403954 10543 webserver.cc:460] Webserver started at http://127.5.122.131:42207/ using document root <none> and password file <none>
I20260620 11:38:08.404217 10543 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:08.404268 10543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:08.404456 10543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:08.405615 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "fd45c8daefa64717baa01fc77a21eb4b"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "4753d65f422530d2543fb96b7e2878b7"
server_key_iv: "f8ff749e5643199ea83122ece244fa5e"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.406124 10543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "fd45c8daefa64717baa01fc77a21eb4b"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "4753d65f422530d2543fb96b7e2878b7"
server_key_iv: "f8ff749e5643199ea83122ece244fa5e"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.407570 10543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:08.408427 10557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.408595 10543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 11:38:08.408710 10543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "fd45c8daefa64717baa01fc77a21eb4b"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "4753d65f422530d2543fb96b7e2878b7"
server_key_iv: "f8ff749e5643199ea83122ece244fa5e"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.408799 10543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:08.412478 10543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:08.412791 10543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:08.412935 10543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:08.413149 10543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:08.413477 10543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:08.413511 10543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.413573 10543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:08.413602 10543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.419505 10543 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:40431
I20260620 11:38:08.419544 10670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:40431 every 8 connection(s)
I20260620 11:38:08.419924 10543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:08.420806 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10543
I20260620 11:38:08.420918 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:08.421129 5610 external_mini_cluster.cc:1428] Setting key 6d79fc75680f1af87e1593415402529d
I20260620 11:38:08.423561 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.420891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:44307 (local address 127.5.122.190:33619)
0620 11:38:08.420954 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.420956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.421254 (+ 298us) server_negotiation.cc:407] Connection header received
0620 11:38:08.421857 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.421858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.421863 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.421902 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.423345 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.423346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.423346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.423351 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.423367 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.423372 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.423434 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.423492 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.423504 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 11:38:08.423825 10673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.420869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:33619 (local address 127.5.122.131:44307)
0620 11:38:08.421111 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.421146 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.421728 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.422056 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.422061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.422071 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.423134 (+ 1063us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.423135 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.423179 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.423218 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.423437 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.423442 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.423444 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.423501 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.423556 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":110,"threads_started":1}
I20260620 11:38:08.424412 10671 heartbeater.cc:347] Connected to a master server at 127.5.122.190:33619
I20260620 11:38:08.424543 10671 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:08.424695 10671 heartbeater.cc:511] Master 127.5.122.190:33619 requested a full tablet report, sending...
I20260620 11:38:08.425084 10221 ts_manager.cc:194] Registered new tserver with Master: fd45c8daefa64717baa01fc77a21eb4b (127.5.122.131:40431)
I20260620 11:38:08.433302 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:38:08.436275 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.434910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33846 (local address 127.5.122.190:33619)
0620 11:38:08.434990 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.434992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.435176 (+ 184us) server_negotiation.cc:407] Connection header received
0620 11:38:08.435208 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.435209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.435216 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.435280 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.436041 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.436042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.436042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.436046 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.436061 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.436066 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.436120 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.436190 (+ 70us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.436205 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 11:38:08.438349 10221 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:08.439922 10221 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:08.444123 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.442302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:33727 (local address 127.0.0.1:33614)
0620 11:38:08.442804 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.442813 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.442860 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.443069 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.443076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.443084 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.443632 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.443633 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.443651 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.443679 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.443985 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.443991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.443994 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.444041 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.444063 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":92,"threads_started":1}
I20260620 11:38:08.444300 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.442290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:43911 (local address 127.0.0.1:53724)
0620 11:38:08.442624 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.442670 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.442916 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.443430 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.443431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.443434 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.443876 (+ 442us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.443877 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.443883 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.443894 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.444140 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.444141 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.444142 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.444192 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.444205 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":105,"threads_started":1}
I20260620 11:38:08.444525 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.442611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33614 (local address 127.5.122.130:33727)
0620 11:38:08.442750 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.442754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.442859 (+ 105us) server_negotiation.cc:407] Connection header received
0620 11:38:08.442933 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.442937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.442948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.443325 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.443847 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.443852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.443852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.443856 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.443891 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.443907 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.444368 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.444395 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.444412 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260620 11:38:08.444572 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.442466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53724 (local address 127.5.122.129:43911)
0620 11:38:08.442677 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.442682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.442750 (+ 68us) server_negotiation.cc:407] Connection header received
0620 11:38:08.442970 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.442976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.442987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.443063 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.443986 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.443995 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.443996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.444000 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.444036 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.444066 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.444147 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.444458 (+ 311us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.444477 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260620 11:38:08.444595 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.442291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:40431 (local address 127.0.0.1:34930)
0620 11:38:08.442765 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.442776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.442833 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.443429 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.443430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.443433 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.444214 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.444215 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.444220 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.444229 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.444479 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.444480 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.444480 (+ 0us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.444524 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.444535 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":114,"threads_started":1}
I20260620 11:38:08.445186 10474 tablet_service.cc:1467] Processing CreateTablet for tablet 9670f602b707482a8f60c5dbb6e76b8e (DEFAULT_TABLE table=test-table [id=4c8388014ff64724859fd7eeb9a6bf3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:08.445207 10343 tablet_service.cc:1467] Processing CreateTablet for tablet 9670f602b707482a8f60c5dbb6e76b8e (DEFAULT_TABLE table=test-table [id=4c8388014ff64724859fd7eeb9a6bf3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:08.445554 10343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9670f602b707482a8f60c5dbb6e76b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:08.445555 10474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9670f602b707482a8f60c5dbb6e76b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:08.448534 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.442522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.5.122.131:40431)
0620 11:38:08.443040 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.443045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.443055 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:08.443078 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.443080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.443088 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.443152 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.444314 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.444323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.444323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.444328 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.444364 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.444398 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.444452 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.444746 (+ 294us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.444766 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":87,"threads_started":1}
I20260620 11:38:08.448536 10685 tablet_bootstrap.cc:492] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Bootstrap starting.
I20260620 11:38:08.449218 10605 tablet_service.cc:1467] Processing CreateTablet for tablet 9670f602b707482a8f60c5dbb6e76b8e (DEFAULT_TABLE table=test-table [id=4c8388014ff64724859fd7eeb9a6bf3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:08.449390 10685 tablet_bootstrap.cc:654] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:08.449551 10605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9670f602b707482a8f60c5dbb6e76b8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:08.449642 10685 log.cc:824] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:08.450016 10686 tablet_bootstrap.cc:492] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7: Bootstrap starting.
I20260620 11:38:08.450481 10685 tablet_bootstrap.cc:492] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: No bootstrap required, opened a new log
I20260620 11:38:08.450552 10685 ts_tablet_manager.cc:1397] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:38:08.450721 10686 tablet_bootstrap.cc:654] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:08.451033 10686 log.cc:824] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:08.451992 10686 tablet_bootstrap.cc:492] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7: No bootstrap required, opened a new log
I20260620 11:38:08.452073 10686 ts_tablet_manager.cc:1397] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:38:08.452768 10689 tablet_bootstrap.cc:492] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Bootstrap starting.
I20260620 11:38:08.453370 10689 tablet_bootstrap.cc:654] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:08.453365 10685 raft_consensus.cc:348] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.453560 10685 raft_consensus.cc:374] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:08.453608 10685 raft_consensus.cc:729] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef603bb2410488e8b300eaf75695fd7, State: Initialized, Role: FOLLOWER
I20260620 11:38:08.453770 10689 log.cc:824] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:08.453770 10685 consensus_queue.cc:260] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.454079 10685 ts_tablet_manager.cc:1428] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 11:38:08.454126 10540 heartbeater.cc:503] Master 127.5.122.190:33619 was elected leader, sending a full tablet report...
I20260620 11:38:08.454746 10689 tablet_bootstrap.cc:492] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: No bootstrap required, opened a new log
I20260620 11:38:08.454838 10689 ts_tablet_manager.cc:1397] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:38:08.454950 10686 raft_consensus.cc:348] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.455143 10686 raft_consensus.cc:374] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:08.455186 10686 raft_consensus.cc:729] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c072588eb444b7b695fd44a1bd46d7, State: Initialized, Role: FOLLOWER
I20260620 11:38:08.455333 10686 consensus_queue.cc:260] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.455627 10686 ts_tablet_manager.cc:1428] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 11:38:08.455642 10409 heartbeater.cc:503] Master 127.5.122.190:33619 was elected leader, sending a full tablet report...
I20260620 11:38:08.457602 10692 raft_consensus.cc:482] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:08.457674 10692 raft_consensus.cc:504] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.457609 10689 raft_consensus.cc:348] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.457808 10689 raft_consensus.cc:374] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:08.457854 10689 raft_consensus.cc:729] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd45c8daefa64717baa01fc77a21eb4b, State: Initialized, Role: FOLLOWER
I20260620 11:38:08.458009 10689 consensus_queue.cc:260] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.458066 10692 leader_election.cc:288] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd45c8daefa64717baa01fc77a21eb4b (127.5.122.131:40431), 7ef603bb2410488e8b300eaf75695fd7 (127.5.122.130:33727)
I20260620 11:38:08.458276 10689 ts_tablet_manager.cc:1428] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 11:38:08.458524 10671 heartbeater.cc:503] Master 127.5.122.190:33619 was elected leader, sending a full tablet report...
I20260620 11:38:08.459561 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.458282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:60099 (local address 127.5.122.131:40431)
0620 11:38:08.458359 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.458363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.458404 (+ 41us) server_negotiation.cc:407] Connection header received
0620 11:38:08.458484 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.458485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.458491 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.458555 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.459289 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.459290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.459290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.459294 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.459309 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.459314 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.459361 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.459478 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.459493 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 11:38:08.459578 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.458213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:40431 (local address 127.5.122.129:60099)
0620 11:38:08.458345 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.458362 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.458416 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.458563 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.458564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.458567 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.459168 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.459168 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.459177 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.459191 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.459397 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.459398 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.459399 (+ 1us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.459471 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.459485 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260620 11:38:08.459887 10625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670f602b707482a8f60c5dbb6e76b8e" candidate_uuid: "44c072588eb444b7b695fd44a1bd46d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd45c8daefa64717baa01fc77a21eb4b" is_pre_election: true
I20260620 11:38:08.460018 10625 raft_consensus.cc:2454] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c072588eb444b7b695fd44a1bd46d7 in term 0.
I20260620 11:38:08.460057 10684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.458471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:53005 (local address 127.5.122.130:33727)
0620 11:38:08.458784 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.458786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.458959 (+ 173us) server_negotiation.cc:407] Connection header received
0620 11:38:08.458988 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.458989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.458994 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.459049 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.459848 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.459849 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.459849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.459854 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.459869 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.459873 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.459931 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.459984 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.459998 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288}
I20260620 11:38:08.460069 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.458247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:33727 (local address 127.5.122.129:53005)
0620 11:38:08.458874 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.458886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.458943 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.459052 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.459052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.459055 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:08.459774 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:08.459774 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:08.459780 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:08.459792 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:08.459929 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:08.459930 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:08.459930 (+ 0us) client_negotiation.cc:769] Sending connection context
0620 11:38:08.459984 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:38:08.459997 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":75,"threads_started":1}
I20260620 11:38:08.460282 10299 leader_election.cc:302] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c072588eb444b7b695fd44a1bd46d7, fd45c8daefa64717baa01fc77a21eb4b; no voters:
I20260620 11:38:08.460314 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670f602b707482a8f60c5dbb6e76b8e" candidate_uuid: "44c072588eb444b7b695fd44a1bd46d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef603bb2410488e8b300eaf75695fd7" is_pre_election: true
I20260620 11:38:08.460435 10692 raft_consensus.cc:2781] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:08.460469 10494 raft_consensus.cc:2454] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c072588eb444b7b695fd44a1bd46d7 in term 0.
I20260620 11:38:08.460500 10692 raft_consensus.cc:482] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:08.460533 10692 raft_consensus.cc:3037] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:08.461318 10692 raft_consensus.cc:504] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.461467 10692 leader_election.cc:288] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [CANDIDATE]: Term 1 election: Requested vote from peers fd45c8daefa64717baa01fc77a21eb4b (127.5.122.131:40431), 7ef603bb2410488e8b300eaf75695fd7 (127.5.122.130:33727)
I20260620 11:38:08.461661 10494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670f602b707482a8f60c5dbb6e76b8e" candidate_uuid: "44c072588eb444b7b695fd44a1bd46d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef603bb2410488e8b300eaf75695fd7"
I20260620 11:38:08.461665 10625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670f602b707482a8f60c5dbb6e76b8e" candidate_uuid: "44c072588eb444b7b695fd44a1bd46d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd45c8daefa64717baa01fc77a21eb4b"
I20260620 11:38:08.461745 10625 raft_consensus.cc:3037] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:08.461745 10494 raft_consensus.cc:3037] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:08.462646 10625 raft_consensus.cc:2454] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c072588eb444b7b695fd44a1bd46d7 in term 1.
I20260620 11:38:08.462646 10494 raft_consensus.cc:2454] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c072588eb444b7b695fd44a1bd46d7 in term 1.
I20260620 11:38:08.462889 10299 leader_election.cc:302] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c072588eb444b7b695fd44a1bd46d7, fd45c8daefa64717baa01fc77a21eb4b; no voters:
I20260620 11:38:08.463001 10692 raft_consensus.cc:2781] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:08.463162 10692 raft_consensus.cc:686] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [term 1 LEADER]: Becoming Leader. State: Replica: 44c072588eb444b7b695fd44a1bd46d7, State: Running, Role: LEADER
I20260620 11:38:08.463266 10692 consensus_queue.cc:237] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } }
I20260620 11:38:08.464141 10221 catalog_manager.cc:5495] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44c072588eb444b7b695fd44a1bd46d7 (127.5.122.129). New cstate: current_term: 1 leader_uuid: "44c072588eb444b7b695fd44a1bd46d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c072588eb444b7b695fd44a1bd46d7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 43911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 } health_report { overall_health: UNKNOWN } } }
I20260620 11:38:08.474454 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.473364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.5.122.129:43911)
0620 11:38:08.473414 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.473416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.473425 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:08.473473 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.473474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.473487 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.473540 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.474246 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.474254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.474254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.474260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.474277 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.474283 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.474336 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.474386 (+ 50us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.474398 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 11:38:08.476670 10494 raft_consensus.cc:1261] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 1 FOLLOWER]: Refusing update from remote peer 44c072588eb444b7b695fd44a1bd46d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:08.476684 10625 raft_consensus.cc:1261] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 1 FOLLOWER]: Refusing update from remote peer 44c072588eb444b7b695fd44a1bd46d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:08.477124 10692 consensus_queue.cc:1035] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd45c8daefa64717baa01fc77a21eb4b" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:08.477267 10695 consensus_queue.cc:1035] T 9670f602b707482a8f60c5dbb6e76b8e P 44c072588eb444b7b695fd44a1bd46d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ef603bb2410488e8b300eaf75695fd7" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:08.480393 10699 mvcc.cc:205] Tried to move back new op lower bound from 7298889680796327936 to 7298889680745668608. Current Snapshot: MvccSnapshot[applied={T|T < 7298889680796327936}]
I20260620 11:38:08.480389 10698 mvcc.cc:205] Tried to move back new op lower bound from 7298889680796327936 to 7298889680745668608. Current Snapshot: MvccSnapshot[applied={T|T < 7298889680796327936 or (T in {7298889680796327936})}]
I20260620 11:38:08.482076 10700 mvcc.cc:205] Tried to move back new op lower bound from 7298889680796327936 to 7298889680745668608. Current Snapshot: MvccSnapshot[applied={T|T < 7298889680796327936 or (T in {7298889680796327936})}]
I20260620 11:38:08.482486 10220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:08.482570 10220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:08.483455 10220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90eed14f51b54cbb8f6f0f2c590fe2bb: Sending DeleteTablet for 3 replicas of tablet 9670f602b707482a8f60c5dbb6e76b8e
I20260620 11:38:08.483773 10343 tablet_service.cc:1514] Processing DeleteTablet for tablet 9670f602b707482a8f60c5dbb6e76b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:08 UTC) from {username='slave'} at 127.0.0.1:53724
I20260620 11:38:08.483804 10605 tablet_service.cc:1514] Processing DeleteTablet for tablet 9670f602b707482a8f60c5dbb6e76b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:08 UTC) from {username='slave'} at 127.0.0.1:34930
I20260620 11:38:08.484015 10474 tablet_service.cc:1514] Processing DeleteTablet for tablet 9670f602b707482a8f60c5dbb6e76b8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:08 UTC) from {username='slave'} at 127.0.0.1:33614
I20260620 11:38:08.484248 10709 tablet_replica.cc:330] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: stopping tablet replica
I20260620 11:38:08.484261 10708 tablet_replica.cc:330] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: stopping tablet replica
I20260620 11:38:08.484355 10708 raft_consensus.cc:2229] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:08.484355 10709 raft_consensus.cc:2229] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:08.484457 10708 raft_consensus.cc:2258] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:08.484457 10709 raft_consensus.cc:2258] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:08.484498 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10281
I20260620 11:38:08.485688 10709 ts_tablet_manager.cc:1905] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:08.486514 10708 ts_tablet_manager.cc:1905] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:08.488523 10708 ts_tablet_manager.cc:1918] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:08.488608 10708 log.cc:1196] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/9670f602b707482a8f60c5dbb6e76b8e
I20260620 11:38:08.488866 10708 ts_tablet_manager.cc:1939] T 9670f602b707482a8f60c5dbb6e76b8e P fd45c8daefa64717baa01fc77a21eb4b: Deleting consensus metadata
I20260620 11:38:08.490163 10208 catalog_manager.cc:4841] TS fd45c8daefa64717baa01fc77a21eb4b (127.5.122.131:40431): tablet 9670f602b707482a8f60c5dbb6e76b8e (table test-table [id=4c8388014ff64724859fd7eeb9a6bf3d]) successfully deleted
I20260620 11:38:08.490540 10709 ts_tablet_manager.cc:1918] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:08.490628 10709 log.cc:1196] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/9670f602b707482a8f60c5dbb6e76b8e
W20260620 11:38:08.490706 10205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:43911 (error 108)
I20260620 11:38:08.490912 10709 ts_tablet_manager.cc:1939] T 9670f602b707482a8f60c5dbb6e76b8e P 7ef603bb2410488e8b300eaf75695fd7: Deleting consensus metadata
I20260620 11:38:08.490942 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10412
I20260620 11:38:08.492089 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.491931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:43911 (local address 127.0.0.1:53748)
0620 11:38:08.492001 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.492052 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:43911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 11:38:08.492285 10205 catalog_manager.cc:4584] TS 44c072588eb444b7b695fd44a1bd46d7 (127.5.122.129:43911): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9670f602b707482a8f60c5dbb6e76b8e: Network error: Client connection negotiation failed: client connection to 127.5.122.129:43911: connect: Connection refused (error 111)
I20260620 11:38:08.497056 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10543
I20260620 11:38:08.497150 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.496970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:33727 (local address 127.0.0.1:33626)
0620 11:38:08.497047 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.497079 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.130:33727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 11:38:08.502146 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10190
2026-06-20T11:38:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (626 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 11:38:08.522797 5610 mini_oidc.cc:152] Starting JWKS server
I20260620 11:38:08.523895 5610 webserver.cc:460] Webserver started at https://127.0.0.1:32987/ using document root <none> and password file <none>
I20260620 11:38:08.524019 5610 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 11:38:08.524206 5610 webserver.cc:460] Webserver started at http://127.0.0.1:41895/ using document root <none> and password file <none>
2026-06-20T11:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:08Z Disabled control of system clock
I20260620 11:38:08.533209 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:44467
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:38099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:44467
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32987/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/testchainca.cert with env {}
W20260620 11:38:08.617313 10716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:08.617568 10716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:08.617594 10716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:08.619005 10716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:08.619067 10716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:08.619081 10716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:08.619093 10716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:08.619107 10716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 11:38:08.619118 10716 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 11:38:08.619130 10716 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32987/jwks.json
I20260620 11:38:08.620946 10716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:38099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:44467
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:44467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32987/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:08.621285 10716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:08.621556 10716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:08.626971 10722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.626888 10724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.626895 10721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:08.627166 10716 server_base.cc:1034] running on GCE node
I20260620 11:38:08.627411 10716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:08.627745 10716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:08.628896 10716 hybrid_clock.cc:648] HybridClock initialized: now 1781955488628875 us; error 43 us; skew 500 ppm
I20260620 11:38:08.630336 10716 webserver.cc:460] Webserver started at http://127.5.122.190:40627/ using document root <none> and password file <none>
I20260620 11:38:08.630611 10716 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:08.630658 10716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:08.630821 10716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:08.631938 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "0fcd2012be004017a979f979c5a8d3e2"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "f586f2d579e88a6f949d26108f1843c3"
server_key_iv: "da640726c2cc7c06eeb9c6c53672afc5"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.632344 10716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "0fcd2012be004017a979f979c5a8d3e2"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "f586f2d579e88a6f949d26108f1843c3"
server_key_iv: "da640726c2cc7c06eeb9c6c53672afc5"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.633708 10716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:08.634593 10730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.634783 10716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:38:08.634893 10716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "0fcd2012be004017a979f979c5a8d3e2"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "f586f2d579e88a6f949d26108f1843c3"
server_key_iv: "da640726c2cc7c06eeb9c6c53672afc5"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.634984 10716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:08.645113 10716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:08.645431 10716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:08.645591 10716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:08.649840 10716 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:44467
I20260620 11:38:08.649875 10782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:44467 every 8 connection(s)
I20260620 11:38:08.650278 10716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:08.651204 10783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:08.653517 10783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2: Bootstrap starting.
I20260620 11:38:08.654182 10783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:08.654489 10783 log.cc:824] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:08.655171 10783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2: No bootstrap required, opened a new log
I20260620 11:38:08.656780 10783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd2012be004017a979f979c5a8d3e2" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44467 } }
I20260620 11:38:08.656944 10783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:08.656968 10783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcd2012be004017a979f979c5a8d3e2, State: Initialized, Role: FOLLOWER
I20260620 11:38:08.657161 10783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd2012be004017a979f979c5a8d3e2" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44467 } }
I20260620 11:38:08.657224 10783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:08.657286 10783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:08.657341 10783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:08.657928 10783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd2012be004017a979f979c5a8d3e2" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44467 } }
I20260620 11:38:08.658110 10783 leader_election.cc:302] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fcd2012be004017a979f979c5a8d3e2; no voters:
I20260620 11:38:08.658315 10783 leader_election.cc:288] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:08.658418 10786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:08.658551 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10716
I20260620 11:38:08.658599 10783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:08.658655 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:08.658653 10786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [term 1 LEADER]: Becoming Leader. State: Replica: 0fcd2012be004017a979f979c5a8d3e2, State: Running, Role: LEADER
I20260620 11:38:08.658754 10786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd2012be004017a979f979c5a8d3e2" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44467 } }
I20260620 11:38:08.658851 5610 external_mini_cluster.cc:1428] Setting key dfacd8ff53c2a045beb70c3aa53269e9
I20260620 11:38:08.659230 10788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fcd2012be004017a979f979c5a8d3e2. Latest consensus state: current_term: 1 leader_uuid: "0fcd2012be004017a979f979c5a8d3e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd2012be004017a979f979c5a8d3e2" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44467 } } }
I20260620 11:38:08.659328 10788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:08.659600 10796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:08.659843 10787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fcd2012be004017a979f979c5a8d3e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fcd2012be004017a979f979c5a8d3e2" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44467 } } }
I20260620 11:38:08.660010 10787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:08.660141 10796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:08.662026 10796 catalog_manager.cc:1347] Generated new cluster ID: f4a7b4ca58cf4104a0de44e76f084183
I20260620 11:38:08.662108 10796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:08.665894 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.659578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.5.122.190:44467)
0620 11:38:08.659790 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.659799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.660190 (+ 391us) server_negotiation.cc:407] Connection header received
0620 11:38:08.660426 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.660435 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.660513 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.660663 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.660953 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.662375 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.663512 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.664175 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.664325 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:08.665106 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.665110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.665110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.665341 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.665391 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.665407 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.665483 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.665566 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.665655 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260620 11:38:08.667042 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:38:08.674240 10796 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:08.674839 10796 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:08.690061 10796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fcd2012be004017a979f979c5a8d3e2: Generated new TSK 0
I20260620 11:38:08.690368 10796 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 11:38:08.716627 10813 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 11:38:08.716778 10813 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 11:38:08.716991 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.700593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35822 (local address 127.5.122.190:44467)
0620 11:38:08.700649 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.700651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.700795 (+ 144us) server_negotiation.cc:407] Connection header received
0620 11:38:08.700865 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.700866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.700879 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.700976 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0620 11:38:08.701133 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.701937 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.702801 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.703129 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.703213 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:08.703488 (+ 275us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 11:38:08.716750 (+ 13262us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 11:38:08.716915 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":30}
W20260620 11:38:08.717061 10797 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35822: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 11:38:08.717514 10812 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:08.700498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44467 (local address 127.0.0.1:35822)
0620 11:38:08.700695 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.700712 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.700777 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.700982 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.700982 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.700984 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 11:38:08.701042 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:08.701043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.702070 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.702071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:08.702693 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:08.702695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.703247 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.703248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:08.703306 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:08.703331 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 11:38:08.717012 (+ 13681us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 11:38:08.717179 (+ 167us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.190:44467: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.122.190:44467: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32987/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 11:38:08.718122 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10716
2026-06-20T11:38:08Z chronyd exiting
I20260620 11:38:08.754448 5610 test_util.cc:175] -----------------------------------------------
I20260620 11:38:08.754534 5610 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781955466149030-5610-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (233 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 11:38:08.755638 5610 mini_oidc.cc:152] Starting JWKS server
I20260620 11:38:08.756639 5610 webserver.cc:460] Webserver started at https://127.0.0.1:40593/ using document root <none> and password file <none>
I20260620 11:38:08.756699 5610 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 11:38:08.756880 5610 webserver.cc:460] Webserver started at http://127.0.0.1:35181/ using document root <none> and password file <none>
2026-06-20T11:38:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:08Z Disabled control of system clock
I20260620 11:38:08.765291 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36969
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:36969
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40593/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/testchainca.cert with env {}
W20260620 11:38:08.850123 10819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:08.850419 10819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:08.850462 10819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:08.851820 10819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:08.851969 10819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:08.852006 10819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:08.852036 10819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:08.852064 10819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 11:38:08.852084 10819 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 11:38:08.852104 10819 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40593/jwks.json
I20260620 11:38:08.853866 10819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:36969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40593/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:08.854308 10819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:08.854619 10819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:38:08.860822 10819 server_base.cc:1034] running on GCE node
W20260620 11:38:08.860839 10827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.860809 10825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:08.860808 10824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:08.861353 10819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:08.861685 10819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:08.862867 10819 hybrid_clock.cc:648] HybridClock initialized: now 1781955488862840 us; error 38 us; skew 500 ppm
I20260620 11:38:08.864528 10819 webserver.cc:460] Webserver started at http://127.5.122.190:39257/ using document root <none> and password file <none>
I20260620 11:38:08.864774 10819 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:08.864825 10819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:08.864928 10819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:08.866236 10819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "008b0b3171e54f799908ee096568a695"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "48ea55eb99a91bef5585f377a5a59ef3"
server_key_iv: "971e82eb425f4b3717107df50f83eac2"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.866590 10819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "008b0b3171e54f799908ee096568a695"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "48ea55eb99a91bef5585f377a5a59ef3"
server_key_iv: "971e82eb425f4b3717107df50f83eac2"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.867923 10819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:08.868687 10833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:08.868913 10819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:08.869011 10819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "008b0b3171e54f799908ee096568a695"
format_stamp: "Formatted at 2026-06-20 11:38:08 on dist-test-slave-szdr"
server_key: "48ea55eb99a91bef5585f377a5a59ef3"
server_key_iv: "971e82eb425f4b3717107df50f83eac2"
server_key_version: "encryptionkey@0"
I20260620 11:38:08.869071 10819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:08.887310 10819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:08.887662 10819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:08.887794 10819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:08.891819 10819 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:36969
I20260620 11:38:08.891856 10885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:36969 every 8 connection(s)
I20260620 11:38:08.892292 10819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:08.893428 10886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:08.896032 10886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695: Bootstrap starting.
I20260620 11:38:08.896701 10886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:08.897033 10886 log.cc:824] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:08.897836 10886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695: No bootstrap required, opened a new log
I20260620 11:38:08.899645 10886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "008b0b3171e54f799908ee096568a695" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36969 } }
I20260620 11:38:08.899852 10886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:08.899901 10886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008b0b3171e54f799908ee096568a695, State: Initialized, Role: FOLLOWER
I20260620 11:38:08.900108 10886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "008b0b3171e54f799908ee096568a695" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36969 } }
I20260620 11:38:08.900223 10886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:08.900270 10886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:08.900345 10886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:08.900584 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10819
I20260620 11:38:08.900682 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:08.900907 5610 external_mini_cluster.cc:1428] Setting key 62c07fc1b38331c57fafd95d8f8fb4d9
I20260620 11:38:08.901165 10886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "008b0b3171e54f799908ee096568a695" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36969 } }
I20260620 11:38:08.901298 10886 leader_election.cc:302] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 008b0b3171e54f799908ee096568a695; no voters:
I20260620 11:38:08.901502 10886 leader_election.cc:288] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:08.901551 10890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:08.901676 10890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [term 1 LEADER]: Becoming Leader. State: Replica: 008b0b3171e54f799908ee096568a695, State: Running, Role: LEADER
I20260620 11:38:08.901790 10886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:08.901762 10890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "008b0b3171e54f799908ee096568a695" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36969 } }
I20260620 11:38:08.902226 10891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 008b0b3171e54f799908ee096568a695. Latest consensus state: current_term: 1 leader_uuid: "008b0b3171e54f799908ee096568a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "008b0b3171e54f799908ee096568a695" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36969 } } }
I20260620 11:38:08.902371 10891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:08.902348 10892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "008b0b3171e54f799908ee096568a695" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "008b0b3171e54f799908ee096568a695" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36969 } } }
I20260620 11:38:08.902467 10892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:08.903264 10900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:08.903851 10900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:08.905715 10900 catalog_manager.cc:1347] Generated new cluster ID: 4357cb59378342b2b7b54ad5aa573330
I20260620 11:38:08.905777 10900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:08.907881 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.901541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44754 (local address 127.5.122.190:36969)
0620 11:38:08.902217 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.902228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.902267 (+ 39us) server_negotiation.cc:407] Connection header received
0620 11:38:08.902396 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.902404 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.902488 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.902649 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:08.903046 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.904445 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.905493 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.906191 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.906328 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:08.907059 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:08.907063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:08.907063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:08.907284 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:08.907332 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:08.907349 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:08.907436 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:08.907566 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 11:38:08.907657 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":134,"threads_started":1}
I20260620 11:38:08.915356 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:38:08.924404 10900 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:08.925200 10900 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:08.933085 10900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 008b0b3171e54f799908ee096568a695: Generated new TSK 0
I20260620 11:38:08.933353 10900 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 11:38:08.963881 10916 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 11:38:08.963989 10916 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 11:38:08.964322 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:08.941949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.5.122.190:36969)
0620 11:38:08.942021 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:08.942023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:08.942140 (+ 117us) server_negotiation.cc:407] Connection header received
0620 11:38:08.942216 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:08.942217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:08.942229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:08.942296 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0620 11:38:08.942464 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.942981 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.944180 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.944384 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.944436 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:08.944654 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 11:38:08.964051 (+ 19397us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 11:38:08.964255 (+ 204us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260620 11:38:08.964421 10893 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 11:38:08.964421 10915 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:08.941834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:36969 (local address 127.0.0.1:44756)
0620 11:38:08.942048 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:08.942061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:08.942138 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:08.942322 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:08.942323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:08.942325 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 11:38:08.942391 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:08.942394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.943079 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.943080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:08.944075 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:08.944079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:08.944469 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:08.944470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:08.944535 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:08.944549 (+ 14us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 11:38:08.964194 (+ 19645us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 11:38:08.964320 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.190:36969: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40593/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":130,"threads_started":1}
I20260620 11:38:08.964864 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10819
2026-06-20T11:38:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 11:38:09.002911 5610 mini_oidc.cc:152] Starting JWKS server
I20260620 11:38:09.003664 5610 webserver.cc:460] Webserver started at https://127.0.0.1:45115/ using document root <none> and password file <none>
I20260620 11:38:09.003712 5610 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 11:38:09.003890 5610 webserver.cc:460] Webserver started at http://127.0.0.1:34245/ using document root <none> and password file <none>
2026-06-20T11:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:09Z Disabled control of system clock
I20260620 11:38:09.012506 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39131
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:39131
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45115/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 11:38:09.098042 10922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:09.098282 10922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:09.098305 10922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:09.099517 10922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:09.099560 10922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:09.099572 10922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:09.099586 10922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:09.099598 10922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 11:38:09.099610 10922 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 11:38:09.099622 10922 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45115/jwks.json
I20260620 11:38:09.101013 10922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:39131
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45115/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:09.101238 10922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:09.101493 10922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:09.106787 10930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:09.106871 10927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:09.106895 10928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:09.106978 10922 server_base.cc:1034] running on GCE node
I20260620 11:38:09.107277 10922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:09.107659 10922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:09.108832 10922 hybrid_clock.cc:648] HybridClock initialized: now 1781955489108806 us; error 43 us; skew 500 ppm
I20260620 11:38:09.110244 10922 webserver.cc:460] Webserver started at http://127.5.122.190:38483/ using document root <none> and password file <none>
I20260620 11:38:09.110519 10922 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:09.110567 10922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:09.110786 10922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:09.111936 10922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "43c0de4862a548bb9e7de389104a6613"
format_stamp: "Formatted at 2026-06-20 11:38:09 on dist-test-slave-szdr"
server_key: "39bf40383263b5e64babcf3824a8f78f"
server_key_iv: "904e1d4e37b94a8aa4042b5869ea070d"
server_key_version: "encryptionkey@0"
I20260620 11:38:09.112361 10922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "43c0de4862a548bb9e7de389104a6613"
format_stamp: "Formatted at 2026-06-20 11:38:09 on dist-test-slave-szdr"
server_key: "39bf40383263b5e64babcf3824a8f78f"
server_key_iv: "904e1d4e37b94a8aa4042b5869ea070d"
server_key_version: "encryptionkey@0"
I20260620 11:38:09.113754 10922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:09.114632 10936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:09.114840 10922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:09.114917 10922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "43c0de4862a548bb9e7de389104a6613"
format_stamp: "Formatted at 2026-06-20 11:38:09 on dist-test-slave-szdr"
server_key: "39bf40383263b5e64babcf3824a8f78f"
server_key_iv: "904e1d4e37b94a8aa4042b5869ea070d"
server_key_version: "encryptionkey@0"
I20260620 11:38:09.115020 10922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:09.124320 10922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:09.124689 10922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:09.124857 10922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:09.129192 10922 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:39131
I20260620 11:38:09.129223 10988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:39131 every 8 connection(s)
I20260620 11:38:09.129609 10922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:09.130499 10989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:09.133526 10989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613: Bootstrap starting.
I20260620 11:38:09.134203 10989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:09.134526 10989 log.cc:824] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:09.135370 10989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613: No bootstrap required, opened a new log
I20260620 11:38:09.137072 10989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c0de4862a548bb9e7de389104a6613" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39131 } }
I20260620 11:38:09.137246 10989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:09.137324 10989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c0de4862a548bb9e7de389104a6613, State: Initialized, Role: FOLLOWER
I20260620 11:38:09.137496 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 10922
I20260620 11:38:09.137509 10989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c0de4862a548bb9e7de389104a6613" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39131 } }
I20260620 11:38:09.137607 10989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:09.137615 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:09.137645 10989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:09.137701 10989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:09.137799 5610 external_mini_cluster.cc:1428] Setting key 13956a1218499fcc6181e5120e82dda5
I20260620 11:38:09.138515 10989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c0de4862a548bb9e7de389104a6613" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39131 } }
I20260620 11:38:09.138648 10989 leader_election.cc:302] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43c0de4862a548bb9e7de389104a6613; no voters:
I20260620 11:38:09.138851 10989 leader_election.cc:288] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:09.138900 10994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:09.139050 10994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [term 1 LEADER]: Becoming Leader. State: Replica: 43c0de4862a548bb9e7de389104a6613, State: Running, Role: LEADER
I20260620 11:38:09.139143 10989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:09.139149 10994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c0de4862a548bb9e7de389104a6613" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39131 } }
I20260620 11:38:09.139525 10995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43c0de4862a548bb9e7de389104a6613" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c0de4862a548bb9e7de389104a6613" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39131 } } }
I20260620 11:38:09.139634 10995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:09.139822 10996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43c0de4862a548bb9e7de389104a6613. Latest consensus state: current_term: 1 leader_uuid: "43c0de4862a548bb9e7de389104a6613" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43c0de4862a548bb9e7de389104a6613" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39131 } } }
I20260620 11:38:09.139889 10996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:09.139871 11000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:09.140635 11000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:09.144968 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:09.138394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.5.122.190:39131)
0620 11:38:09.138636 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:09.138646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:09.138679 (+ 33us) server_negotiation.cc:407] Connection header received
0620 11:38:09.138817 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:09.138822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:09.138927 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:09.139113 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:09.139515 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.141438 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.142613 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.143224 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.143364 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:09.144133 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:09.144137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:09.144138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:09.144350 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:09.144393 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:09.144410 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:09.144483 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:09.144635 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 11:38:09.144726 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
I20260620 11:38:09.146293 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:38:09.147285 11000 catalog_manager.cc:1347] Generated new cluster ID: d816174d0bbf4e208dbebf6d1f40adbe
I20260620 11:38:09.147342 11000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:09.168154 11000 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:09.168746 11000 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:09.174574 11000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43c0de4862a548bb9e7de389104a6613: Generated new TSK 0
I20260620 11:38:09.174813 11000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:38:09.188931 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:09.184762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.5.122.190:39131)
0620 11:38:09.184820 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:09.184823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:09.184949 (+ 126us) server_negotiation.cc:407] Connection header received
0620 11:38:09.184991 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:09.184992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:09.185006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:09.185090 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0620 11:38:09.185352 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.186179 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.187015 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.187348 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.187422 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:09.187647 (+ 225us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 11:38:09.188714 (+ 1067us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45115/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 11:38:09.188854 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60760: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45115/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":34}
W20260620 11:38:09.189004 10993 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60760: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45115/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 11:38:09.190917 11018 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:09.184673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39131 (local address 127.0.0.1:60760)
0620 11:38:09.184856 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:09.184867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:09.184938 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:09.185201 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:09.185202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:09.185204 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 11:38:09.185262 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:09.185264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.186277 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.186278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:09.186918 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:09.186921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.187459 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.187460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:09.187522 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:09.187528 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 11:38:09.188831 (+ 1303us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45115/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 11:38:09.190809 (+ 1978us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.190:39131: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45115/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":1754,"thread_start_us":91,"threads_started":1}
I20260620 11:38:09.191179 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 10922
E20260620 11:38:09.201507 11019 webserver.cc:533] Webserver: sslize failed:
E20260620 11:38:09.201592 11019 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-20T11:38:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (226 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T11:38:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:09Z Disabled control of system clock
I20260620 11:38:09.238185 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35869
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:36027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:35869
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/testchainca.cert with env {}
W20260620 11:38:09.325162 11023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:09.325382 11023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:09.325407 11023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:09.326704 11023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:09.326753 11023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:09.326766 11023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:09.326779 11023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:09.326792 11023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 11:38:09.326804 11023 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 11:38:09.326815 11023 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 11:38:09.328346 11023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:35869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:09.328554 11023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:09.328727 11023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:09.333796 11028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:09.333815 11031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:09.333813 11029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:09.333868 11023 server_base.cc:1034] running on GCE node
I20260620 11:38:09.334270 11023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:09.334564 11023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:09.335763 11023 hybrid_clock.cc:648] HybridClock initialized: now 1781955489335749 us; error 32 us; skew 500 ppm
I20260620 11:38:09.337191 11023 webserver.cc:460] Webserver started at http://127.5.122.190:40041/ using document root <none> and password file <none>
I20260620 11:38:09.337502 11023 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:09.337589 11023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:09.337747 11023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:09.338968 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "f34ae73efeab40a4a9c139c797d86e5c"
format_stamp: "Formatted at 2026-06-20 11:38:09 on dist-test-slave-szdr"
server_key: "15ae8e98f1f2b85dc2cf2ec48d63c9a7"
server_key_iv: "f1a399715d637381ad08e034d2095dbf"
server_key_version: "encryptionkey@0"
I20260620 11:38:09.339375 11023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "f34ae73efeab40a4a9c139c797d86e5c"
format_stamp: "Formatted at 2026-06-20 11:38:09 on dist-test-slave-szdr"
server_key: "15ae8e98f1f2b85dc2cf2ec48d63c9a7"
server_key_iv: "f1a399715d637381ad08e034d2095dbf"
server_key_version: "encryptionkey@0"
I20260620 11:38:09.340764 11023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:38:09.341565 11037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:09.341761 11023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:09.341872 11023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "f34ae73efeab40a4a9c139c797d86e5c"
format_stamp: "Formatted at 2026-06-20 11:38:09 on dist-test-slave-szdr"
server_key: "15ae8e98f1f2b85dc2cf2ec48d63c9a7"
server_key_iv: "f1a399715d637381ad08e034d2095dbf"
server_key_version: "encryptionkey@0"
I20260620 11:38:09.341959 11023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:09.352806 11023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:09.353175 11023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:09.353348 11023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:09.357533 11023 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:35869
I20260620 11:38:09.357599 11089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:35869 every 8 connection(s)
I20260620 11:38:09.358099 11023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:09.359150 11090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:09.361552 11090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c: Bootstrap starting.
I20260620 11:38:09.362236 11090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:09.362561 11090 log.cc:824] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:09.362924 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11023
I20260620 11:38:09.363056 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:09.363255 5610 external_mini_cluster.cc:1428] Setting key 3f84a4b2dbd89277e8e504eea749e38d
I20260620 11:38:09.363431 11090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c: No bootstrap required, opened a new log
I20260620 11:38:09.366493 11090 raft_consensus.cc:348] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34ae73efeab40a4a9c139c797d86e5c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35869 } }
I20260620 11:38:09.366679 11090 raft_consensus.cc:374] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:09.366710 11090 raft_consensus.cc:729] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34ae73efeab40a4a9c139c797d86e5c, State: Initialized, Role: FOLLOWER
I20260620 11:38:09.366828 11090 consensus_queue.cc:260] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34ae73efeab40a4a9c139c797d86e5c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35869 } }
I20260620 11:38:09.366899 11090 raft_consensus.cc:388] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:09.366935 11090 raft_consensus.cc:482] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:09.366993 11090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:09.367786 11090 raft_consensus.cc:504] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34ae73efeab40a4a9c139c797d86e5c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35869 } }
I20260620 11:38:09.367908 11090 leader_election.cc:302] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f34ae73efeab40a4a9c139c797d86e5c; no voters:
I20260620 11:38:09.368094 11090 leader_election.cc:288] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:09.368130 11095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:09.368259 11095 raft_consensus.cc:686] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [term 1 LEADER]: Becoming Leader. State: Replica: f34ae73efeab40a4a9c139c797d86e5c, State: Running, Role: LEADER
I20260620 11:38:09.368347 11095 consensus_queue.cc:237] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34ae73efeab40a4a9c139c797d86e5c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35869 } }
I20260620 11:38:09.368424 11090 sys_catalog.cc:564] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:09.368654 11095 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f34ae73efeab40a4a9c139c797d86e5c. Latest consensus state: current_term: 1 leader_uuid: "f34ae73efeab40a4a9c139c797d86e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34ae73efeab40a4a9c139c797d86e5c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35869 } } }
I20260620 11:38:09.368829 11095 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:09.368664 11096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f34ae73efeab40a4a9c139c797d86e5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f34ae73efeab40a4a9c139c797d86e5c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35869 } } }
I20260620 11:38:09.369537 11096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:09.369549 11101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:09.370290 11101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:09.370493 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:09.363797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42164 (local address 127.5.122.190:35869)
0620 11:38:09.364033 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:09.364043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:09.364078 (+ 35us) server_negotiation.cc:407] Connection header received
0620 11:38:09.364916 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:09.364927 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:09.365011 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:09.365175 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:09.365412 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.366894 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.368032 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.368714 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.369255 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:09.369646 (+ 391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:09.369651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:09.369652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:09.369922 (+ 270us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:09.370003 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:09.370020 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:09.370086 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:09.370170 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 11:38:09.370256 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260620 11:38:09.371640 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:38:09.372309 11101 catalog_manager.cc:1347] Generated new cluster ID: 22b9214cd12f41dbba382d2062c952c6
I20260620 11:38:09.372357 11101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:09.377871 11101 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:09.378445 11101 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:09.382833 11101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f34ae73efeab40a4a9c139c797d86e5c: Generated new TSK 0
I20260620 11:38:09.383045 11101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:38:09.442046 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:09.391851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.5.122.190:35869)
0620 11:38:09.391904 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:09.391906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:09.391977 (+ 71us) server_negotiation.cc:407] Connection header received
0620 11:38:09.392071 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:09.392072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:09.392083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:09.392146 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0620 11:38:09.392341 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.392857 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.394081 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.394336 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.394396 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:09.394631 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 11:38:09.441752 (+ 47121us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 11:38:09.441910 (+ 158us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42180: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 11:38:09.442145 11094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42180: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 11:38:09.442219 11119 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:09.391720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35869 (local address 127.0.0.1:42180)
0620 11:38:09.391911 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:09.391923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:09.392000 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:09.392193 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:09.392194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:09.392196 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0620 11:38:09.392265 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:09.392267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.392964 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.392966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:09.393997 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:09.394001 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:09.394449 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:09.394450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:09.394517 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:09.394525 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 11:38:09.441926 (+ 47401us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 11:38:09.442101 (+ 175us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.122.190:35869: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260620 11:38:09.442667 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11023
2026-06-20T11:38:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (230 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:09 dist-test-slave-szdr krb5kdc[11131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:09 dist-test-slave-szdr krb5kdc[11131](info): set up 2 sockets
Jun 20 11:38:09 dist-test-slave-szdr krb5kdc[11131](info): commencing operation
krb5kdc: starting...
W20260620 11:38:10.678287 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:10 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:10.804899 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41967
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:10.890480 11144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:10.890723 11144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:10.890748 11144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:10.892145 11144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:10.892200 11144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:10.892215 11144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:10.892228 11144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:10.892241 11144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:10.894305 11144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:10.894618 11144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:10.894877 11144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:10.900241 11149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:10.900264 11144 server_base.cc:1034] running on GCE node
W20260620 11:38:10.900259 11152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:10.900244 11150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:10.900805 11144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:10.901180 11144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:10.902362 11144 hybrid_clock.cc:648] HybridClock initialized: now 1781955490902349 us; error 50 us; skew 500 ppm
Jun 20 11:38:10 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955490, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:10.904978 11144 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:10.905457 11144 webserver.cc:460] Webserver started at http://127.5.122.190:33427/ using document root <none> and password file <none>
I20260620 11:38:10.905710 11144 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:10.905788 11144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:10.905951 11144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:10.907125 11144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "7839047052fa490399ebda75467f68f5"
format_stamp: "Formatted at 2026-06-20 11:38:10 on dist-test-slave-szdr"
server_key: "9edab2e0cf151fb94f32432d85dbf48b"
server_key_iv: "0e300459b1ac34cf5cc59c3d006348aa"
server_key_version: "encryptionkey@0"
I20260620 11:38:10.907577 11144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "7839047052fa490399ebda75467f68f5"
format_stamp: "Formatted at 2026-06-20 11:38:10 on dist-test-slave-szdr"
server_key: "9edab2e0cf151fb94f32432d85dbf48b"
server_key_iv: "0e300459b1ac34cf5cc59c3d006348aa"
server_key_version: "encryptionkey@0"
I20260620 11:38:10.908986 11144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:10.909822 11159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:10.910077 11144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:10.910192 11144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "7839047052fa490399ebda75467f68f5"
format_stamp: "Formatted at 2026-06-20 11:38:10 on dist-test-slave-szdr"
server_key: "9edab2e0cf151fb94f32432d85dbf48b"
server_key_iv: "0e300459b1ac34cf5cc59c3d006348aa"
server_key_version: "encryptionkey@0"
I20260620 11:38:10.910282 11144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:10.928193 11144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:10.930337 11144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:10.930537 11144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:10.934791 11144 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:41967
I20260620 11:38:10.934829 11211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:41967 every 8 connection(s)
I20260620 11:38:10.935256 11144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:10.936172 11212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:10.938725 11212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5: Bootstrap starting.
I20260620 11:38:10.939356 11212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:10.939670 11212 log.cc:824] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:10.940367 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11144
I20260620 11:38:10.940492 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:10.940464 11212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5: No bootstrap required, opened a new log
I20260620 11:38:10.940709 5610 external_mini_cluster.cc:1428] Setting key b4f098cae53f359365186907aff1dea1
I20260620 11:38:10.942945 11212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839047052fa490399ebda75467f68f5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41967 } }
I20260620 11:38:10.943260 11212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:10.943352 11212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7839047052fa490399ebda75467f68f5, State: Initialized, Role: FOLLOWER
I20260620 11:38:10.943550 11212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839047052fa490399ebda75467f68f5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41967 } }
I20260620 11:38:10.943642 11212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:10.943684 11212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:10.943723 11212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:10.944489 11212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839047052fa490399ebda75467f68f5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41967 } }
I20260620 11:38:10.944599 11212 leader_election.cc:302] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7839047052fa490399ebda75467f68f5; no voters:
I20260620 11:38:10.944778 11212 leader_election.cc:288] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:10.944862 11217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:10.945017 11217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [term 1 LEADER]: Becoming Leader. State: Replica: 7839047052fa490399ebda75467f68f5, State: Running, Role: LEADER
I20260620 11:38:10.945096 11212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:10.945156 11217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839047052fa490399ebda75467f68f5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41967 } }
I20260620 11:38:10.945613 11217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7839047052fa490399ebda75467f68f5. Latest consensus state: current_term: 1 leader_uuid: "7839047052fa490399ebda75467f68f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839047052fa490399ebda75467f68f5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41967 } } }
I20260620 11:38:10.945833 11217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:10.945761 11219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7839047052fa490399ebda75467f68f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7839047052fa490399ebda75467f68f5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41967 } } }
I20260620 11:38:10.946250 11219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:10.946457 11226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:10.947212 11226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:10.948941 11226 catalog_manager.cc:1347] Generated new cluster ID: 39e509a3696d4589a41eb3a961e97eec
I20260620 11:38:10.948992 11226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:38:10 dist-test-slave-szdr krb5kdc[11131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:10.953241 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:10.941325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.5.122.190:41967)
0620 11:38:10.941542 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:10.941552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:10.941588 (+ 36us) server_negotiation.cc:407] Connection header received
0620 11:38:10.942140 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:10.942158 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:10.942231 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:10.942398 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:10.943130 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:10.944869 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:10.946057 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:10.946776 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:10.946912 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:10.949511 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:10.949516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:10.949516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:10.949530 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:10.951416 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:10.951845 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:10.951848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:10.951849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:10.951947 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:10.952179 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:10.952180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:10.952180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:10.952438 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:10.952626 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:10.952697 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 11:38:10.952845 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260620 11:38:10.954654 11226 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:10.955284 11226 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:10.961257 11226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7839047052fa490399ebda75467f68f5: Generated new TSK 0
I20260620 11:38:10.961475 11226 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:38:10.993587 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41967
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:11.088230 11238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:11.088477 11238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:11.088505 11238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:11.090010 11238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:11.090067 11238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:11.090102 11238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:11.091710 11238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:11.091995 11238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:11.092296 11238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:11.097627 11243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:11.097621 11246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:11.097627 11244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:11.097918 11238 server_base.cc:1034] running on GCE node
I20260620 11:38:11.098148 11238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:11.098439 11238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:11.099643 11238 hybrid_clock.cc:648] HybridClock initialized: now 1781955491099632 us; error 29 us; skew 500 ppm
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.102761 11238 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:38:11.103324 11238 webserver.cc:460] Webserver started at http://127.5.122.129:35519/ using document root <none> and password file <none>
I20260620 11:38:11.103585 11238 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:11.103632 11238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:11.103812 11238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:11.104948 11238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "6f410102b251436bb2577b7eb804e574"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "417c7d9551bcd9571d0f491c4aba0192"
server_key_iv: "a3d0c1e3a2f49ff15f83903a148d1e39"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.105334 11238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "6f410102b251436bb2577b7eb804e574"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "417c7d9551bcd9571d0f491c4aba0192"
server_key_iv: "a3d0c1e3a2f49ff15f83903a148d1e39"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.106892 11238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:11.107676 11253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.107883 11238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:11.107998 11238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "6f410102b251436bb2577b7eb804e574"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "417c7d9551bcd9571d0f491c4aba0192"
server_key_iv: "a3d0c1e3a2f49ff15f83903a148d1e39"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.108084 11238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:11.124513 11238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:11.127310 11238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:11.127506 11238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:11.127805 11238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:11.128149 11238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:11.128211 11238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.128258 11238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:11.128281 11238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.134725 11238 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:36121
I20260620 11:38:11.134790 11366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:36121 every 8 connection(s)
I20260620 11:38:11.135177 11238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:11.139782 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11238
I20260620 11:38:11.139890 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:11.140105 5610 external_mini_cluster.cc:1428] Setting key 6b5657bf7b96f37d3725633660902bb8
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:11.146092 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.136041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:46749 (local address 127.5.122.190:41967)
0620 11:38:11.136155 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.136157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.136354 (+ 197us) server_negotiation.cc:407] Connection header received
0620 11:38:11.137405 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.137407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.137421 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.137472 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.138297 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.138835 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.140009 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.140310 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.140407 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.142915 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:11.142918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:11.142918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:11.142930 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:11.144478 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.144942 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.144943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.144943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.145015 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.145315 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.145316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.145317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.145455 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:11.145539 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:11.145843 (+ 304us) server_negotiation.cc:299] Negotiation successful
0620 11:38:11.145993 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 11:38:11.146248 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.136010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41967 (local address 127.5.122.129:46749)
0620 11:38:11.136205 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:11.136258 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:11.137259 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:11.137548 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:11.137552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:11.137837 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:11.138180 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:11.138186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.138943 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.138944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:11.139895 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:11.139898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.140418 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.140419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:11.140562 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.141151 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:11.141160 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:11.142804 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:11.144577 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:11.144586 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:11.144589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:11.144842 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:11.145105 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:11.145106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:11.145107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:11.145240 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:11.145530 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:11.145537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:11.145685 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 11:38:11.145799 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 11:38:11.146005 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260620 11:38:11.146873 11367 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41967
I20260620 11:38:11.146992 11367 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:11.147213 11367 heartbeater.cc:511] Master 127.5.122.190:41967 requested a full tablet report, sending...
I20260620 11:38:11.147809 11176 ts_manager.cc:194] Registered new tserver with Master: 6f410102b251436bb2577b7eb804e574 (127.5.122.129:36121)
I20260620 11:38:11.148516 11176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:46749
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:38:11.168996 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41967
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:11.253569 11372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:11.253796 11372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:11.253829 11372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:11.255144 11372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:11.255191 11372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:11.255220 11372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:11.256793 11372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:11.257231 11372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:11.257557 11372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:11.262833 11380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:11.262846 11377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:11.262847 11378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:11.262881 11372 server_base.cc:1034] running on GCE node
I20260620 11:38:11.263235 11372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:11.263530 11372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:11.264711 11372 hybrid_clock.cc:648] HybridClock initialized: now 1781955491264690 us; error 44 us; skew 500 ppm
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.267467 11372 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:38:11.267939 11372 webserver.cc:460] Webserver started at http://127.5.122.130:37955/ using document root <none> and password file <none>
I20260620 11:38:11.268217 11372 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:11.268294 11372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:11.268441 11372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:11.269512 11372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "456623935c5e41f9b833d8d9c0a55f99"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "081ce4e6b9b499f135a3173bca364991"
server_key_iv: "9d18bb7af86e6be2fd9c9feccca3814a"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.269892 11372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "456623935c5e41f9b833d8d9c0a55f99"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "081ce4e6b9b499f135a3173bca364991"
server_key_iv: "9d18bb7af86e6be2fd9c9feccca3814a"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.271414 11372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:11.272214 11387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.272415 11372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.272518 11372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "456623935c5e41f9b833d8d9c0a55f99"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "081ce4e6b9b499f135a3173bca364991"
server_key_iv: "9d18bb7af86e6be2fd9c9feccca3814a"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.272605 11372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:11.287717 11372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:11.289829 11372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:11.290071 11372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:11.290376 11372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:11.290721 11372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:11.290787 11372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.290831 11372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:11.290853 11372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.297012 11372 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:46773
I20260620 11:38:11.297057 11500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:46773 every 8 connection(s)
I20260620 11:38:11.297456 11372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:11.304396 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11372
I20260620 11:38:11.304539 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:11.304745 5610 external_mini_cluster.cc:1428] Setting key 2236cecc939eb3db1f893d11e01c63bb
I20260620 11:38:11.308142 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.298358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:51505 (local address 127.5.122.190:41967)
0620 11:38:11.298429 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.298431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.298640 (+ 209us) server_negotiation.cc:407] Connection header received
0620 11:38:11.299586 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.299587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.299597 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.299660 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.300289 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.300790 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.301863 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.302079 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.302136 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.304498 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:11.304501 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:11.304501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:11.304513 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:11.306571 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.307030 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.307032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.307032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.307111 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.307453 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.307453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.307454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.307564 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:11.307618 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:11.308002 (+ 384us) server_negotiation.cc:299] Negotiation successful
0620 11:38:11.308073 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:38:11.308467 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.298336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41967 (local address 127.5.122.130:51505)
0620 11:38:11.298498 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:11.298543 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:11.299486 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:11.299718 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:11.299722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:11.299991 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:11.300199 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:11.300203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.300892 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.300893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:11.301761 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:11.301763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.302158 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.302158 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:11.302294 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.302802 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:11.302811 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:11.304379 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:11.306668 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:11.306678 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:11.306680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:11.306941 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:11.307210 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:11.307211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:11.307211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:11.307375 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:11.307633 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:11.307639 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:11.307855 (+ 216us) client_negotiation.cc:769] Sending connection context
0620 11:38:11.307981 (+ 126us) client_negotiation.cc:240] Negotiation successful
0620 11:38:11.308190 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260620 11:38:11.309029 11501 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41967
I20260620 11:38:11.309141 11501 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:11.309353 11501 heartbeater.cc:511] Master 127.5.122.190:41967 requested a full tablet report, sending...
I20260620 11:38:11.309823 11176 ts_manager.cc:194] Registered new tserver with Master: 456623935c5e41f9b833d8d9c0a55f99 (127.5.122.130:46773)
I20260620 11:38:11.310272 11176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:51505
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:38:11.332486 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41967
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:11.416412 11506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:11.416692 11506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:11.416738 11506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:11.418041 11506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:11.418126 11506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:11.418174 11506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:11.419790 11506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:11.420101 11506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:11.420356 11506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:11.425770 11511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:11.425770 11514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:11.425793 11512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:11.426169 11506 server_base.cc:1034] running on GCE node
I20260620 11:38:11.426448 11506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:11.426728 11506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:11.427910 11506 hybrid_clock.cc:648] HybridClock initialized: now 1781955491427892 us; error 53 us; skew 500 ppm
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.430652 11506 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:38:11.431092 11506 webserver.cc:460] Webserver started at http://127.5.122.131:37533/ using document root <none> and password file <none>
I20260620 11:38:11.431288 11506 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:11.431329 11506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:11.431437 11506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:11.432588 11506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "d2def210402c4ddb9ae5e6590f055731"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "7000dbf3d564a0131bb9f835f7e08bdc"
server_key_iv: "2f9183aa3e78bb88fa511eaa4bcf50e8"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.432904 11506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "d2def210402c4ddb9ae5e6590f055731"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "7000dbf3d564a0131bb9f835f7e08bdc"
server_key_iv: "2f9183aa3e78bb88fa511eaa4bcf50e8"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.434412 11506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:11.435168 11521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.435388 11506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:11.435485 11506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "d2def210402c4ddb9ae5e6590f055731"
format_stamp: "Formatted at 2026-06-20 11:38:11 on dist-test-slave-szdr"
server_key: "7000dbf3d564a0131bb9f835f7e08bdc"
server_key_iv: "2f9183aa3e78bb88fa511eaa4bcf50e8"
server_key_version: "encryptionkey@0"
I20260620 11:38:11.435596 11506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:11.449035 11506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:11.451319 11506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:11.451509 11506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:11.451778 11506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:11.452127 11506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:11.452167 11506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.452191 11506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:11.452246 11506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:11.458535 11506 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:40489
I20260620 11:38:11.458576 11634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:40489 every 8 connection(s)
I20260620 11:38:11.459018 11506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:11.467670 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11506
I20260620 11:38:11.467782 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:11.467980 5610 external_mini_cluster.cc:1428] Setting key 5a2af1d9ff4e8a393193d21fddcaa1f6
I20260620 11:38:11.469071 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.459842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:44261 (local address 127.5.122.190:41967)
0620 11:38:11.459885 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.459890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.460097 (+ 207us) server_negotiation.cc:407] Connection header received
0620 11:38:11.461091 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.461092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.461105 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.461224 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.461871 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.462457 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.463528 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.463719 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.463783 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.466152 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:11.466153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:11.466154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:11.466161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:11.467362 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.467759 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.467760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.467761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.467829 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.468139 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.468140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.468140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.468346 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:11.468431 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:11.468865 (+ 434us) server_negotiation.cc:299] Negotiation successful
0620 11:38:11.468963 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:38:11.469326 11637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.459800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41967 (local address 127.5.122.131:44261)
0620 11:38:11.459977 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:11.460021 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:11.460970 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:11.461301 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:11.461305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:11.461580 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:11.461782 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:11.461786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.462544 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.462545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:11.463431 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:11.463433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.463806 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.463807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:11.463923 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.464542 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:11.464552 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:11.466065 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:11.467435 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:11.467441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:11.467444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:11.467676 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:11.467909 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:11.467910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:11.467910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:11.468058 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:11.468445 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:11.468453 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:11.468691 (+ 238us) client_negotiation.cc:769] Sending connection context
0620 11:38:11.468820 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 11:38:11.469054 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260620 11:38:11.469789 11635 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41967
I20260620 11:38:11.469918 11635 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:11.470139 11635 heartbeater.cc:511] Master 127.5.122.190:41967 requested a full tablet report, sending...
I20260620 11:38:11.470579 11176 ts_manager.cc:194] Registered new tserver with Master: d2def210402c4ddb9ae5e6590f055731 (127.5.122.131:40489)
I20260620 11:38:11.470940 11176 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:44261
I20260620 11:38:11.480046 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/test-admin.keytab.
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.498396 5610 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 11:38:11.500185 11642 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:11.499349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.5.122.129:36121)
0620 11:38:11.499552 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.499558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.499575 (+ 17us) server_negotiation.cc:407] Connection header received
0620 11:38:11.499600 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.499602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.499616 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.499697 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.500095 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37144: BlockingRecv error: recv got EOF from 127.0.0.1:37144 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
W20260620 11:38:11.500322 5610 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5cc' of type FILE
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.501415 5610 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.502415 5610 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:38:11.510658 11642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.503007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37152 (local address 127.5.122.129:36121)
0620 11:38:11.503072 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.503074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.503123 (+ 49us) server_negotiation.cc:407] Connection header received
0620 11:38:11.503187 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.503187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.503200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.503264 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.503686 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.504434 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.505190 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.505550 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.505649 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.507868 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:11.507887 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:11.507888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:11.507896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:11.509319 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.509726 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.509731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.509731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.509792 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.510038 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.510039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.510039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.510193 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:11.510365 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:11.510436 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 11:38:11.510528 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260620 11:38:11.512658 11642 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:11.512057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.5.122.129:36121)
0620 11:38:11.512111 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.512113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.512166 (+ 53us) server_negotiation.cc:407] Connection header received
0620 11:38:11.512241 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.512242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.512253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.512308 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.512599 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37164: BlockingRecv error: recv got EOF from 127.0.0.1:37164 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260620 11:38:11.512724 5610 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5cc' of type FILE
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.513873 5610 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.514856 5610 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:38:11.522660 11642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.515483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.5.122.129:36121)
0620 11:38:11.515537 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.515539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.515638 (+ 99us) server_negotiation.cc:407] Connection header received
0620 11:38:11.515719 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.515720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.515731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.515802 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.516208 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.516724 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.517459 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.517670 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.517750 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.520327 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:11.520328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:11.520329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:11.520337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:11.521566 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.521907 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.521907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.521908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.522060 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.522237 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.522237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.522238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.522343 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:11.522431 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:11.522523 (+ 92us) server_negotiation.cc:299] Negotiation successful
0620 11:38:11.522590 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260620 11:38:11.524453 11642 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:11.523782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.5.122.129:36121)
0620 11:38:11.523840 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.523842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.523910 (+ 68us) server_negotiation.cc:407] Connection header received
0620 11:38:11.523986 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.523987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.523997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.524045 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.524400 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37176: BlockingRecv error: recv got EOF from 127.0.0.1:37176 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260620 11:38:11.524546 5610 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781955466149030-5610-0/krb5kdc/krb5cc' of type FILE
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.525563 5610 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:11.526485 5610 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11131](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:38:11.533821 11642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:11.527053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.5.122.129:36121)
0620 11:38:11.527111 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:11.527113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:11.527188 (+ 75us) server_negotiation.cc:407] Connection header received
0620 11:38:11.527267 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:11.527268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:11.527278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:11.527326 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:11.527716 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.528196 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.528954 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:11.529149 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:11.529237 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:11.531411 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:11.531412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:11.531413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:11.531421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:11.532633 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.533007 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.533009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.533009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.533069 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:11.533288 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:11.533289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:11.533289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:11.533407 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:11.533529 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:11.533632 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 11:38:11.533716 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:38:11.534611 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11238
I20260620 11:38:11.539499 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11372
I20260620 11:38:11.544670 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11506
I20260620 11:38:11.549057 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11144
2026-06-20T11:38:11Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1920 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11655](info): set up 2 sockets
Jun 20 11:38:11 dist-test-slave-szdr krb5kdc[11655](info): commencing operation
krb5kdc: starting...
W20260620 11:38:12.606189 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:12 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab.
Entry for principal oryx/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab.
I20260620 11:38:12.728658 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38037
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab
--principal=oryx/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:38037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:12.814815 11668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:12.815099 11668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:12.815140 11668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:12.816359 11668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:12.816448 11668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:12.816473 11668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:12.816493 11668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:12.816521 11668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:12.818135 11668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:38037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab
--principal=oryx/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:12.818449 11668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:12.818708 11668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:12.824002 11674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:12.824043 11668 server_base.cc:1034] running on GCE node
W20260620 11:38:12.824023 11676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:12.824023 11673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:12.824486 11668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:12.824805 11668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:12.826013 11668 hybrid_clock.cc:648] HybridClock initialized: now 1781955492826057 us; error 94 us; skew 500 ppm
Jun 20 11:38:12 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955492, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:12.833107 11668 init.cc:377] Logged in from keytab as oryx/127.5.122.190@KRBTEST.COM (short username oryx)
I20260620 11:38:12.833647 11668 webserver.cc:460] Webserver started at http://127.5.122.190:46075/ using document root <none> and password file <none>
I20260620 11:38:12.833935 11668 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:12.834069 11668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:12.834236 11668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:12.835372 11668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "587a77cc787b4b9196ffa4f87575a120"
format_stamp: "Formatted at 2026-06-20 11:38:12 on dist-test-slave-szdr"
server_key: "b823d2e9cb55562c539a38adfc4c0afe"
server_key_iv: "bca91c489b6d05da35facdf9557f23c5"
server_key_version: "encryptionkey@0"
I20260620 11:38:12.835781 11668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "587a77cc787b4b9196ffa4f87575a120"
format_stamp: "Formatted at 2026-06-20 11:38:12 on dist-test-slave-szdr"
server_key: "b823d2e9cb55562c539a38adfc4c0afe"
server_key_iv: "bca91c489b6d05da35facdf9557f23c5"
server_key_version: "encryptionkey@0"
I20260620 11:38:12.837216 11668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:12.838035 11683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:12.838234 11668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:12.838342 11668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "587a77cc787b4b9196ffa4f87575a120"
format_stamp: "Formatted at 2026-06-20 11:38:12 on dist-test-slave-szdr"
server_key: "b823d2e9cb55562c539a38adfc4c0afe"
server_key_iv: "bca91c489b6d05da35facdf9557f23c5"
server_key_version: "encryptionkey@0"
I20260620 11:38:12.838436 11668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:12.850322 11668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:12.852768 11668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:12.852955 11668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:12.857697 11668 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:38037
I20260620 11:38:12.857736 11735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:38037 every 8 connection(s)
I20260620 11:38:12.858232 11668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:12.859395 11736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:12.861825 11736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: Bootstrap starting.
I20260620 11:38:12.862521 11736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:12.862854 11736 log.cc:824] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:12.863665 11736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: No bootstrap required, opened a new log
I20260620 11:38:12.863893 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11668
I20260620 11:38:12.864032 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:12.864248 5610 external_mini_cluster.cc:1428] Setting key 9209f8c3e17f7c0679b01287d66620d4
I20260620 11:38:12.866277 11736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } }
I20260620 11:38:12.866482 11736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:12.866520 11736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 587a77cc787b4b9196ffa4f87575a120, State: Initialized, Role: FOLLOWER
I20260620 11:38:12.866686 11736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } }
I20260620 11:38:12.866758 11736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:12.866780 11736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:12.866853 11736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:12.868068 11736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } }
I20260620 11:38:12.868182 11736 leader_election.cc:302] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 587a77cc787b4b9196ffa4f87575a120; no voters:
I20260620 11:38:12.868340 11736 leader_election.cc:288] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:12.868427 11741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:12.868646 11741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 1 LEADER]: Becoming Leader. State: Replica: 587a77cc787b4b9196ffa4f87575a120, State: Running, Role: LEADER
I20260620 11:38:12.868747 11736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:12.868801 11741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } }
I20260620 11:38:12.870502 11742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } }
I20260620 11:38:12.870652 11743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587a77cc787b4b9196ffa4f87575a120. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } }
I20260620 11:38:12.870653 11742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:12.870765 11743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
W20260620 11:38:12.870994 11756 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:38:12.871412 11756 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:38:12.871526 11758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:12.872754 11758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 11:38:12 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:12.874590 11758 catalog_manager.cc:1347] Generated new cluster ID: 8feadcad4ee24a3cab7552f41b0d395d
I20260620 11:38:12.874678 11758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:12.877614 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:12.864878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57864 (local address 127.5.122.190:38037)
0620 11:38:12.865100 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:12.865110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:12.865146 (+ 36us) server_negotiation.cc:407] Connection header received
0620 11:38:12.866224 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:12.866235 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:12.866311 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:12.866507 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:12.867351 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:12.868993 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:12.870218 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:12.870911 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:12.871029 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:12.873839 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:12.873843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:12.873843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:12.873875 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:12.875839 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:12.876257 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:12.876260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:12.876260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:12.876388 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:12.876653 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:12.876654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:12.876655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:12.876929 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:12.877120 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:12.877242 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 11:38:12.877390 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260620 11:38:12.885493 11758 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:12.886152 11758 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:12.890014 11758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: Generated new TSK 0
I20260620 11:38:12.890237 11758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab.
Entry for principal oryx/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab.
I20260620 11:38:12.926406 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38037
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab
--principal=oryx/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:13.028216 11762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:13.028441 11762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:13.028468 11762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:13.029740 11762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:13.029790 11762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:13.029819 11762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:13.031404 11762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab
--principal=oryx/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:13.031673 11762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:13.031905 11762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:13.037384 11767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.037403 11770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.037386 11768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:13.037417 11762 server_base.cc:1034] running on GCE node
I20260620 11:38:13.037932 11762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:13.038251 11762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:13.039441 11762 hybrid_clock.cc:648] HybridClock initialized: now 1781955493039423 us; error 27 us; skew 500 ppm
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:13.042484 11762 init.cc:377] Logged in from keytab as oryx/127.5.122.129@KRBTEST.COM (short username oryx)
I20260620 11:38:13.042883 11762 webserver.cc:460] Webserver started at http://127.5.122.129:42725/ using document root <none> and password file <none>
I20260620 11:38:13.043131 11762 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:13.043177 11762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:13.043332 11762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:13.044428 11762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "2d80693a30fb469cb49af5d9122d0c73"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "5db2c6ab5eba92a6396b27aec25626d4"
server_key_iv: "632a9245971a6c50fe4fe0ac71a0681d"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.044811 11762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "2d80693a30fb469cb49af5d9122d0c73"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "5db2c6ab5eba92a6396b27aec25626d4"
server_key_iv: "632a9245971a6c50fe4fe0ac71a0681d"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.046207 11762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:13.046963 11777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.047147 11762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:13.047258 11762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "2d80693a30fb469cb49af5d9122d0c73"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "5db2c6ab5eba92a6396b27aec25626d4"
server_key_iv: "632a9245971a6c50fe4fe0ac71a0681d"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.047348 11762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:13.083020 11762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:13.085675 11762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:13.085876 11762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:13.086186 11762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:13.086530 11762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:13.086592 11762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.086639 11762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:13.086665 11762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.092689 11762 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:36139
I20260620 11:38:13.092782 11890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:36139 every 8 connection(s)
I20260620 11:38:13.093174 11762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:13.103397 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11762
I20260620 11:38:13.103535 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:13.103796 5610 external_mini_cluster.cc:1428] Setting key 7798ec817490b88c13410d84e87c0cfe
I20260620 11:38:13.103904 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.094133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:35297 (local address 127.5.122.190:38037)
0620 11:38:13.094178 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.094180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.094546 (+ 366us) server_negotiation.cc:407] Connection header received
0620 11:38:13.095066 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.095068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.095085 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.095184 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.096162 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.096658 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.097693 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.097908 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.098033 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.100327 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.100328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.100328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.100339 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.102199 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.102661 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.102662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.102662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.102727 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.102978 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.102979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.102979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.103123 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.103188 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.103649 (+ 461us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.103785 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 11:38:13.104581 11893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.094101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:35297)
0620 11:38:13.094292 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.094339 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.094853 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.095266 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.095270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.095862 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.096061 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.096065 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.096751 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.096752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.097595 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.097597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.098032 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.098033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.098147 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.098675 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.098684 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.100177 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.102285 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.102294 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.102296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.102557 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.102801 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.102802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.102802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.102913 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.103222 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.103229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.103424 (+ 195us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.104047 (+ 623us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.104297 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260620 11:38:13.105070 11891 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38037
I20260620 11:38:13.105168 11891 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:13.105386 11891 heartbeater.cc:511] Master 127.5.122.190:38037 requested a full tablet report, sending...
I20260620 11:38:13.105916 11700 ts_manager.cc:194] Registered new tserver with Master: 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:13.106570 11700 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:35297
WARNING: no policy specified for oryx/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab.
Entry for principal oryx/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab.
I20260620 11:38:13.130524 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38037
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab
--principal=oryx/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:13.212435 11896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:13.212679 11896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:13.212705 11896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:13.214004 11896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:13.214083 11896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:13.214113 11896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:13.215687 11896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab
--principal=oryx/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:13.215981 11896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:13.216241 11896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:13.221562 11901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.221544 11902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:13.221719 11896 server_base.cc:1034] running on GCE node
W20260620 11:38:13.221596 11904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:13.222052 11896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:13.222364 11896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:13.223528 11896 hybrid_clock.cc:648] HybridClock initialized: now 1781955493223512 us; error 32 us; skew 500 ppm
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:13.226318 11896 init.cc:377] Logged in from keytab as oryx/127.5.122.130@KRBTEST.COM (short username oryx)
I20260620 11:38:13.226802 11896 webserver.cc:460] Webserver started at http://127.5.122.130:33063/ using document root <none> and password file <none>
I20260620 11:38:13.227054 11896 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:13.227128 11896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:13.227288 11896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:13.228379 11896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "089d79306e8d4dfabd921e78de330db9"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "a4309938c9a0074b8a9b955130f53094"
server_key_iv: "eb07b207c52ef38844b75358f47e477d"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.228763 11896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "089d79306e8d4dfabd921e78de330db9"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "a4309938c9a0074b8a9b955130f53094"
server_key_iv: "eb07b207c52ef38844b75358f47e477d"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.230240 11896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:13.231135 11911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.231334 11896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:13.231441 11896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "089d79306e8d4dfabd921e78de330db9"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "a4309938c9a0074b8a9b955130f53094"
server_key_iv: "eb07b207c52ef38844b75358f47e477d"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.231529 11896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:13.240437 11896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:13.243532 11896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:13.243727 11896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:13.243990 11896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:13.244328 11896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:13.244390 11896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.244434 11896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:13.244457 11896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.253576 11896 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:34363
I20260620 11:38:13.253756 12024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:34363 every 8 connection(s)
I20260620 11:38:13.254161 11896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:13.256361 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 11896
I20260620 11:38:13.256493 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:13.256731 5610 external_mini_cluster.cc:1428] Setting key 8e1ab312e38a2d61a0b1bf7b1adf1abe
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:13.265552 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.254994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:43859 (local address 127.5.122.190:38037)
0620 11:38:13.255076 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.255079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.255330 (+ 251us) server_negotiation.cc:407] Connection header received
0620 11:38:13.255842 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.255844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.255859 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.255948 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.257132 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.257779 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.258893 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.259092 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.259186 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.262201 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.262202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.262203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.262213 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.263972 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.264442 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.264443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.264444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.264509 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.264810 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.264811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.264811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.264938 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.265019 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.265354 (+ 335us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.265453 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 11:38:13.265928 12027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.254986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:43859)
0620 11:38:13.255177 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.255223 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.255735 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.256031 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.256034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.256719 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.256984 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.256992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.257887 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.257888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.258789 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.258791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.259193 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.259194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.259336 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.260003 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.260016 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.262088 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.264061 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.264067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.264070 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.264316 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.264584 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.264585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.264586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.264731 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.265009 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.265017 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.265209 (+ 192us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.265334 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.265516 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":118,"threads_started":1}
I20260620 11:38:13.266534 12025 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38037
I20260620 11:38:13.266664 12025 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:13.266863 12025 heartbeater.cc:511] Master 127.5.122.190:38037 requested a full tablet report, sending...
I20260620 11:38:13.267295 11700 ts_manager.cc:194] Registered new tserver with Master: 089d79306e8d4dfabd921e78de330db9 (127.5.122.130:34363)
I20260620 11:38:13.267697 11700 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:43859
WARNING: no policy specified for oryx/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab.
Entry for principal oryx/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab.
I20260620 11:38:13.289690 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38037
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab
--principal=oryx/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:13.375059 12030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:13.375347 12030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:13.375391 12030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:13.376636 12030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:13.376724 12030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:13.376775 12030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:13.378425 12030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab
--principal=oryx/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:13.378737 12030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:13.378997 12030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:13.384219 12036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.384200 12035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.384253 12038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:13.384464 12030 server_base.cc:1034] running on GCE node
I20260620 11:38:13.384671 12030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:13.384972 12030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:13.386148 12030 hybrid_clock.cc:648] HybridClock initialized: now 1781955493386129 us; error 24 us; skew 500 ppm
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:13.388669 12030 init.cc:377] Logged in from keytab as oryx/127.5.122.131@KRBTEST.COM (short username oryx)
I20260620 11:38:13.389086 12030 webserver.cc:460] Webserver started at http://127.5.122.131:36943/ using document root <none> and password file <none>
I20260620 11:38:13.389336 12030 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:13.389385 12030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:13.389549 12030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:13.390946 12030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "9d6bc15329f242cf8a865bb76548b697"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "1430241b61aa396b8d7986b2e2ea4013"
server_key_iv: "d7568b3a1813d03c696d1d2980694053"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.391364 12030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "9d6bc15329f242cf8a865bb76548b697"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "1430241b61aa396b8d7986b2e2ea4013"
server_key_iv: "d7568b3a1813d03c696d1d2980694053"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.392772 12030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:13.393558 12045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.393757 12030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:13.393859 12030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "9d6bc15329f242cf8a865bb76548b697"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "1430241b61aa396b8d7986b2e2ea4013"
server_key_iv: "d7568b3a1813d03c696d1d2980694053"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.393942 12030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:13.413177 12030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:13.415433 12030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:13.415601 12030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:13.415880 12030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:13.416239 12030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:13.416275 12030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.416301 12030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:13.416355 12030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.422571 12030 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:40381
I20260620 11:38:13.422613 12158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:40381 every 8 connection(s)
I20260620 11:38:13.422991 12030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:13.425540 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 12030
I20260620 11:38:13.425658 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:13.425827 5610 external_mini_cluster.cc:1428] Setting key 3e1a0e314b801341a753ac98c8c06a39
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:13.433195 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.423773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:49241 (local address 127.5.122.190:38037)
0620 11:38:13.423878 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.423880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.424078 (+ 198us) server_negotiation.cc:407] Connection header received
0620 11:38:13.424653 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.424654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.424667 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.424716 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.425807 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.426655 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.427782 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.427991 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.428072 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.430420 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.430422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.430422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.430430 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.431908 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.432313 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.432313 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.432314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.432367 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.432618 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.432619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.432619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.432737 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.432789 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.433055 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.433122 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 11:38:13.433493 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.423731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:49241)
0620 11:38:13.423930 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.423983 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.424529 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.424810 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.424815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.425458 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.425715 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.425722 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.426754 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.426756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.427671 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.427674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.428080 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.428081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.428224 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.428738 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.428746 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.430327 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.431993 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.431999 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.432001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.432244 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.432435 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.432436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.432436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.432541 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.432803 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.432808 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.432934 (+ 126us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.433026 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.433195 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":97,"threads_started":1}
I20260620 11:38:13.434085 12159 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38037
I20260620 11:38:13.434221 12159 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:13.434485 12159 heartbeater.cc:511] Master 127.5.122.190:38037 requested a full tablet report, sending...
I20260620 11:38:13.434922 11700 ts_manager.cc:194] Registered new tserver with Master: 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131:40381)
I20260620 11:38:13.435367 11700 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:49241
I20260620 11:38:13.437683 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.122.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.189.keytab.
Entry for principal oryx/127.5.122.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.189.keytab.
I20260620 11:38:13.464864 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:33391
--webserver_interface=127.5.122.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.189.keytab
--principal=oryx/127.5.122.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:38037,127.5.122.189:33391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:13.547842 12164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:13.548080 12164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:13.548105 12164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:13.549345 12164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:13.549394 12164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:13.549409 12164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:13.549422 12164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:13.549487 12164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:13.551105 12164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:38037,127.5.122.189:33391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.189.keytab
--principal=oryx/127.5.122.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:33391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
W20260620 11:38:13.551223 12164 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.122.190:38037,127.5.122.189:33391'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 11:38:13.551342 12164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:13.551534 12164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:38:13.556828 12164 server_base.cc:1034] running on GCE node
W20260620 11:38:13.556802 12169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.556792 12172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.556774 12170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:13.557274 12164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:13.557581 12164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:13.558730 12164 hybrid_clock.cc:648] HybridClock initialized: now 1781955493558698 us; error 49 us; skew 500 ppm
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:13.561316 12164 init.cc:377] Logged in from keytab as oryx/127.5.122.189@KRBTEST.COM (short username oryx)
I20260620 11:38:13.561825 12164 webserver.cc:460] Webserver started at http://127.5.122.189:38595/ using document root <none> and password file <none>
I20260620 11:38:13.562125 12164 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:13.562194 12164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:13.562335 12164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:13.563511 12164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/instance:
uuid: "207ee50512ab407abb46e19afe1e2291"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "244b63654e3cc4dcb2f58140d26b45ec"
server_key_iv: "13ebf2364bf435516ebe112c54cf6861"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.563896 12164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal/instance:
uuid: "207ee50512ab407abb46e19afe1e2291"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "244b63654e3cc4dcb2f58140d26b45ec"
server_key_iv: "13ebf2364bf435516ebe112c54cf6861"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.565289 12164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:13.566157 12179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.566349 12164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:13.566463 12164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
uuid: "207ee50512ab407abb46e19afe1e2291"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "244b63654e3cc4dcb2f58140d26b45ec"
server_key_iv: "13ebf2364bf435516ebe112c54cf6861"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.566553 12164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:13.573705 12164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:13.575680 12164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:13.575868 12164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:13.580458 12164 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.189:33391
I20260620 11:38:13.580499 12231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.189:33391 every 8 connection(s)
I20260620 11:38:13.580933 12164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
I20260620 11:38:13.581900 12232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:13.583550 12232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.189@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:13.590194 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 12164
I20260620 11:38:13.593410 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.584337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.5.122.190:38037)
0620 11:38:13.584415 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.584417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.584643 (+ 226us) server_negotiation.cc:407] Connection header received
0620 11:38:13.585152 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.585153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.585163 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.585223 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.585902 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.586455 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.587507 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.587700 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.587756 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.589914 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.589916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.589916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.589923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.591835 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.592344 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.592346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.592346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.592408 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.592689 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.592690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.592690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.592832 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.592883 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.593225 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.593311 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:38:13.593528 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.584297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:57872)
0620 11:38:13.584498 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.584548 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.585061 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.585298 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.585302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.585585 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.585820 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.585825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.586566 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.586567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.587417 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.587419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.587780 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.587781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.587886 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.588383 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.588391 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.589804 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.591949 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.591956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.591958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.592262 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.592487 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.592488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.592488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.592625 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.592895 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.592899 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.593056 (+ 157us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.593187 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.593318 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
I20260620 11:38:13.594343 12232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.189@KRBTEST.COM
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.189@KRBTEST.COM for oryx/127.5.122.189@KRBTEST.COM
I20260620 11:38:13.600741 12236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.590750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57590 (local address 127.5.122.189:33391)
0620 11:38:13.591067 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.591071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.591086 (+ 15us) server_negotiation.cc:407] Connection header received
0620 11:38:13.591103 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.591105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.591115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.591169 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.591982 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.593007 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.594147 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.594587 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.594654 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.597632 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.597639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.597639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.597645 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.599382 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.599767 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.599771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.599771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.599834 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.600054 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.600055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.600056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.600205 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.600392 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.600446 (+ 54us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.600547 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
I20260620 11:38:13.602677 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.594615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.0.0.1:57592)
0620 11:38:13.594660 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.594671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.594703 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.594927 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.594928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.595092 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.595140 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.595141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.596132 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.596133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.596808 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.596811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.597253 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.597254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.597320 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.597846 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.597850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.599264 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.601656 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.601657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.601657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.601821 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.602052 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.602053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.602054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.602108 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.602357 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.602358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.602396 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.602466 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.602567 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260620 11:38:13.602757 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.594663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.5.122.189:33391)
0620 11:38:13.594804 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.594806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.594812 (+ 6us) server_negotiation.cc:407] Connection header received
0620 11:38:13.594833 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.594834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.594841 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.594926 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.595224 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.596057 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.596895 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.597159 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.597253 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.599351 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.599353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.599353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.599360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.601557 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.601896 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.601897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.601897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.602001 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.602181 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.602182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.602182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.602290 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.602366 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.602497 (+ 131us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.602617 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
I20260620 11:38:13.604091 12232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Bootstrap starting.
I20260620 11:38:13.604766 12232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:13.605074 12232 log.cc:824] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:13.605827 12232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: No bootstrap required, opened a new log
I20260620 11:38:13.607334 12232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } }
I20260620 11:38:13.607496 12232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:13.607544 12232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207ee50512ab407abb46e19afe1e2291, State: Initialized, Role: FOLLOWER
I20260620 11:38:13.607699 12232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } }
I20260620 11:38:13.608016 12232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:13.607981 12239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } } }
I20260620 11:38:13.608251 12239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 11:38:13.609742 12250 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:38:13.609807 12250 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:38:13.611444 12164 master_runner.cc:428] Detected that this master 207ee50512ab407abb46e19afe1e2291 is joining an existing cluster
I20260620 11:38:13.611496 12164 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.122.189:33391
I20260620 11:38:13.619648 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.612722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57880 (local address 127.5.122.190:38037)
0620 11:38:13.612769 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.612772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.613007 (+ 235us) server_negotiation.cc:407] Connection header received
0620 11:38:13.613083 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.613084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.613096 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.613153 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.613564 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.614165 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.614942 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.615156 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.615245 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.616438 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.616439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.616440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.616448 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.618251 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.618698 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.618699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.618700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.618758 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.619008 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.619009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.619010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.619148 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.619204 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.619434 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.619522 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:38:13.619810 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.612746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.5.122.189:33391)
0620 11:38:13.612785 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.612786 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.613032 (+ 246us) server_negotiation.cc:407] Connection header received
0620 11:38:13.613120 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.613121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.613129 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.613155 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.613622 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.614394 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.615400 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.615661 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.615755 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.616677 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.616678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.616679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.616685 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.618561 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.618948 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.618949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.618950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.618997 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.619223 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.619224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.619225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.619343 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.619418 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.619588 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.619728 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":18}
I20260620 11:38:13.619963 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.612649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:57880)
0620 11:38:13.612943 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.612956 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.613019 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.613157 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.613157 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.613400 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.613468 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.613470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.614274 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.614275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.614844 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.614846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.615245 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.615246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.615319 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.615892 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.615898 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.616338 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.618345 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.618346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.618347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.618640 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.618845 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.618845 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.618846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.618926 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.619254 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.619255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.619311 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.619743 (+ 432us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.619835 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":221,"threads_started":1}
I20260620 11:38:13.620419 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.612684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.0.0.1:57606)
0620 11:38:13.612968 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.612978 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.613054 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.613209 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.613210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.613465 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.613533 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.613536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.614497 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.614498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.615313 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.615315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.615794 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.615795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.615874 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.616276 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.616281 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.616596 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.618650 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.618651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.618651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.618874 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.619071 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.619072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.619072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.619148 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.619417 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.619418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.619474 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.619550 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.619664 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"spinlock_wait_cycles":251136,"thread_start_us":242,"threads_started":1}
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955492, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.190@KRBTEST.COM for oryx/127.5.122.189@KRBTEST.COM
I20260620 11:38:13.630298 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.622596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.0.0.1:57610)
0620 11:38:13.622780 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.622826 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.622900 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.623043 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.623046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.623300 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.623350 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.623352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.624253 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.624254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.624848 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.624850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.625308 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.625310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.625381 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.625898 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.625903 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.627543 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.629199 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.629207 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.629209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.629418 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.629630 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.629631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.629631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.629685 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.629941 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.629947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.630050 (+ 103us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.630120 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.630220 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260620 11:38:13.630360 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.622659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57610 (local address 127.5.122.189:33391)
0620 11:38:13.622721 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.622723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.622904 (+ 181us) server_negotiation.cc:407] Connection header received
0620 11:38:13.622959 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.622960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.622970 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.623041 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.623424 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.624171 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.624932 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.625230 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.625303 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.627633 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.627635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.627635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.627643 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.629113 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.629498 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.629499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.629500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.629562 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.629746 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.629747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.629747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.629878 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.629945 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.630181 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.630266 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 11:38:13.630954 11700 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.122.189:33391: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "207ee50512ab407abb46e19afe1e2291"
member_type: NON_VOTER
last_known_addr {
host: "127.5.122.189"
port: 33391
}
attrs {
promote: true
}
}
dest_uuid: "587a77cc787b4b9196ffa4f87575a120"
cas_config_opid_index: -1
I20260620 11:38:13.631301 11700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } }
I20260620 11:38:13.631812 12260 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 207ee50512ab407abb46e19afe1e2291 (127.5.122.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } }
I20260620 11:38:13.632154 12206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:13.632925 12260 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.122.189:33391
I20260620 11:38:13.632959 12206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Refusing update from remote peer 587a77cc787b4b9196ffa4f87575a120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 11:38:13.633050 12260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } }
I20260620 11:38:13.633114 12260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:13.633198 12164 master.cc:537] Master@127.5.122.189:33391 shutting down...
I20260620 11:38:13.633411 12259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 11:38:13.634280 11684 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260620 11:38:13.634752 11684 consensus_peers.cc:487] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 -> Peer 207ee50512ab407abb46e19afe1e2291 (127.5.122.189:33391): Couldn't send request to peer 207ee50512ab407abb46e19afe1e2291. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 11:38:13.636421 12164 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:13.636520 12164 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:13.636571 12164 tablet_replica.cc:330] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: stopping tablet replica
I20260620 11:38:13.649395 12164 master.cc:552] Master@127.5.122.189:33391 shutdown complete.
I20260620 11:38:13.649483 12164 master_runner.cc:305] Clearing existing system tablet
I20260620 11:38:13.649622 12164 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:13.649689 12164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:13.650588 12164 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:13.651124 12264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.651278 12164 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:13.651432 12164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:38:13.651482 12164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
uuid: "207ee50512ab407abb46e19afe1e2291"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "244b63654e3cc4dcb2f58140d26b45ec"
server_key_iv: "13ebf2364bf435516ebe112c54cf6861"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.652318 12164 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:13.654103 12164 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 11:38:13.654183 12164 log.cc:1196] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 11:38:13.654408 12164 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Deleting consensus metadata
I20260620 11:38:13.654544 12164 master_runner.cc:315] Copying system tablet from 127.5.122.190:38037
I20260620 11:38:13.655154 12164 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: tablet copy: Beginning tablet copy session from remote peer at address 127.5.122.190:38037
I20260620 11:38:13.661723 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.655552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:57884)
0620 11:38:13.655712 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.655730 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.655785 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.655945 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.655945 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.656099 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.656156 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.656158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.657127 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.657128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.657658 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.657659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.658187 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.658188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.658237 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.658751 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.658755 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.659061 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.660770 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.660771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.660771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.660945 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.661158 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.661158 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.661159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.661208 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.661472 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.661472 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.661508 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.661559 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.661639 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260620 11:38:13.661762 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.655660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57884 (local address 127.5.122.190:38037)
0620 11:38:13.655728 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.655731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.655776 (+ 45us) server_negotiation.cc:407] Connection header received
0620 11:38:13.655833 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.655834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.655845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.655922 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.656233 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.657044 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.657747 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.658114 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.658188 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.659155 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.659156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.659157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.659164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.660690 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.661024 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.661025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.661026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.661091 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.661272 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.661273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.661273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.661408 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.661485 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.661586 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.661668 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:38:13.662254 11720 tablet_copy_service.cc:140] P 587a77cc787b4b9196ffa4f87575a120: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 207ee50512ab407abb46e19afe1e2291 ({username='oryx', principal='oryx/127.5.122.189@KRBTEST.COM'} at 127.0.0.1:57884)
I20260620 11:38:13.662366 11720 tablet_copy_service.cc:161] P 587a77cc787b4b9196ffa4f87575a120: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 207ee50512ab407abb46e19afe1e2291 at {username='oryx', principal='oryx/127.5.122.189@KRBTEST.COM'} at 127.0.0.1:57884: session id = 207ee50512ab407abb46e19afe1e2291-00000000000000000000000000000000
I20260620 11:38:13.663030 11720 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 11:38:13.663734 12164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 11:38:13.665066 12164 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: tablet copy: Starting download of 0 data blocks...
I20260620 11:38:13.665226 12164 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: tablet copy: Starting download of 1 WAL segments...
I20260620 11:38:13.666152 12164 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 11:38:13.667126 11720 tablet_copy_service.cc:342] P 587a77cc787b4b9196ffa4f87575a120: Request end of tablet copy session 207ee50512ab407abb46e19afe1e2291-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.122.189@KRBTEST.COM'} at 127.0.0.1:57884
I20260620 11:38:13.667189 11720 tablet_copy_service.cc:433] P 587a77cc787b4b9196ffa4f87575a120: ending tablet copy session 207ee50512ab407abb46e19afe1e2291-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 207ee50512ab407abb46e19afe1e2291
I20260620 11:38:13.670017 12164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:13.671263 12272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.671299 12271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.671346 12274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:13.674153 12164 server_base.cc:1034] running on GCE node
I20260620 11:38:13.674369 12164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:13.674602 12164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:13.675755 12164 hybrid_clock.cc:648] HybridClock initialized: now 1781955493675746 us; error 31 us; skew 500 ppm
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:13.677394 12164 init.cc:377] Logged in from keytab as oryx/127.5.122.189@KRBTEST.COM (short username oryx)
I20260620 11:38:13.677716 12164 webserver.cc:460] Webserver started at http://127.5.122.189:34289/ using document root <none> and password file <none>
I20260620 11:38:13.677863 12164 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:13.677919 12164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:13.678648 12164 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.679217 12281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.679384 12164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 11:38:13.679459 12164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
uuid: "207ee50512ab407abb46e19afe1e2291"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "244b63654e3cc4dcb2f58140d26b45ec"
server_key_iv: "13ebf2364bf435516ebe112c54cf6861"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.679528 12164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:13.700930 12164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:13.702978 12164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:13.706248 12164 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.189:33391
I20260620 11:38:13.706271 12333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.189:33391 every 8 connection(s)
I20260620 11:38:13.706622 12164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
I20260620 11:38:13.707399 12334 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:38:13.707821 12334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Bootstrap starting.
I20260620 11:38:13.710144 12334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:38:13.710394 12334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Bootstrap complete.
I20260620 11:38:13.710613 12334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } }
I20260620 11:38:13.710713 12334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 207ee50512ab407abb46e19afe1e2291, State: Initialized, Role: LEARNER
I20260620 11:38:13.710757 12334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } }
I20260620 11:38:13.710979 12334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:13.710964 12337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } }
I20260620 11:38:13.711061 12337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: LEARNER
I20260620 11:38:13.712574 12348 catalog_manager.cc:1259] Loaded cluster ID: 8feadcad4ee24a3cab7552f41b0d395d
I20260620 11:38:13.712633 12348 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: loading cluster ID for follower catalog manager: success
I20260620 11:38:13.713604 12348 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: acquiring CA information for follower catalog manager: success
I20260620 11:38:13.713923 12348 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 11:38:13.760102 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.752982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.5.122.189:33391)
0620 11:38:13.753142 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.753144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.753160 (+ 16us) server_negotiation.cc:407] Connection header received
0620 11:38:13.753186 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.753186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.753196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.753284 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.753890 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.754651 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.755531 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.755745 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.755832 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.757189 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.757191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.757191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.757200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.758833 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.759247 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.759248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.759249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.759313 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.759538 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.759539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.759539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.759659 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.759774 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.759883 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.759986 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for oryx/127.5.122.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.188.keytab.
Entry for principal oryx/127.5.122.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.188.keytab.
I20260620 11:38:13.785844 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:40581
--webserver_interface=127.5.122.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.188.keytab
--principal=oryx/127.5.122.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:13.871835 12352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:13.872062 12352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:13.872087 12352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:13.873332 12352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:13.873378 12352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:13.873395 12352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:13.873438 12352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:13.873466 12352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:13.875149 12352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.188.keytab
--principal=oryx/127.5.122.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:40581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:13.875437 12352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:13.875696 12352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:13.880918 12357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.880945 12358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:13.881417 12360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:13.881559 12352 server_base.cc:1034] running on GCE node
I20260620 11:38:13.881848 12352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:13.882192 12352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:13.883373 12352 hybrid_clock.cc:648] HybridClock initialized: now 1781955493883360 us; error 26 us; skew 500 ppm
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:13.886027 12352 init.cc:377] Logged in from keytab as oryx/127.5.122.188@KRBTEST.COM (short username oryx)
I20260620 11:38:13.886528 12352 webserver.cc:460] Webserver started at http://127.5.122.188:39915/ using document root <none> and password file <none>
I20260620 11:38:13.886775 12352 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:13.886821 12352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:13.887006 12352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:13.888154 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/instance:
uuid: "1bb8b2a0560640c092885231201d3923"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "436edbe1af5efe648d9beafcb350eff2"
server_key_iv: "53138df11ecdcf8216cbe13abefffc14"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.888561 12352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal/instance:
uuid: "1bb8b2a0560640c092885231201d3923"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "436edbe1af5efe648d9beafcb350eff2"
server_key_iv: "53138df11ecdcf8216cbe13abefffc14"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.890046 12352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:13.890882 12367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:13.891083 12352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:13.891194 12352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
uuid: "1bb8b2a0560640c092885231201d3923"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "436edbe1af5efe648d9beafcb350eff2"
server_key_iv: "53138df11ecdcf8216cbe13abefffc14"
server_key_version: "encryptionkey@0"
I20260620 11:38:13.891276 12352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:13.904728 12352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:13.906805 12352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:13.906980 12352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:13.911511 12352 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.188:40581
I20260620 11:38:13.911605 12419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.188:40581 every 8 connection(s)
I20260620 11:38:13.911998 12352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
I20260620 11:38:13.913122 12420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:13.914834 12420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.188@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:13.920850 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 12352
I20260620 11:38:13.924499 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.915629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.5.122.190:38037)
0620 11:38:13.915709 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.915711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.915827 (+ 116us) server_negotiation.cc:407] Connection header received
0620 11:38:13.916309 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.916310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.916319 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.916381 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.916997 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.917503 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.918528 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.918727 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.918785 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.921157 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.921159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.921159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.921174 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.922923 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.923374 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.923376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.923376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.923456 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.923843 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.923843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.923844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.923963 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.924017 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.924355 (+ 338us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.924429 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:38:13.924731 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.915500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:57900)
0620 11:38:13.915683 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.915727 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.916218 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.916457 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.916460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.916713 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.916917 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.916921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.917590 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.917590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.918440 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.918443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.918821 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.918822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.918978 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.919487 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.919503 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.921045 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.923029 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.923036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.923037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.923276 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.923535 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.923536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.923536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.923734 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.924033 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.924039 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.924202 (+ 163us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.924323 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.924491 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":119,"threads_started":1}
I20260620 11:38:13.925585 12420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.188@KRBTEST.COM
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.188@KRBTEST.COM for oryx/127.5.122.189@KRBTEST.COM
I20260620 11:38:13.931241 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.921730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.5.122.188:40581)
0620 11:38:13.921902 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.921907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.921924 (+ 17us) server_negotiation.cc:407] Connection header received
0620 11:38:13.921941 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.921943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.921954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.922077 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.922796 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.923707 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.924730 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.925228 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.925304 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.928058 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.928065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.928065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.928071 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.929894 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.930315 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.930318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.930319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.930395 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.930592 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.930592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.930593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.930774 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.930963 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.931022 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.931102 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260620 11:38:13.933465 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.925869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.0.0.1:57638)
0620 11:38:13.925903 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.926835 (+ 932us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.926879 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.927033 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.927033 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.927147 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.927194 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.927195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.927982 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.927983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.928705 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.928708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.929118 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.929119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.929194 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.929578 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.929582 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.931064 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.932680 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.932681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.932682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.932843 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.933016 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.933017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.933017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.933065 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.933246 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.933246 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.933280 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.933330 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.933400 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260620 11:38:13.933478 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.926012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57638 (local address 127.5.122.189:33391)
0620 11:38:13.926087 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.926089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.926917 (+ 828us) server_negotiation.cc:407] Connection header received
0620 11:38:13.926947 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.926948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.926966 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.927032 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.927278 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.927877 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.928808 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.929025 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.929119 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.931165 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.931167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.931168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.931176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.932606 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.932908 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.932909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.932909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.932959 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.933103 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.933103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.933103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.933194 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.933239 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.933346 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.933413 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 11:38:13.933946 12420 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955493, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.188@KRBTEST.COM for oryx/127.5.122.188@KRBTEST.COM
I20260620 11:38:13.940224 12422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.934268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.0.0.1:35830)
0620 11:38:13.934321 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.934332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.934369 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.934501 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.934502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.934644 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.934672 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.934673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.935385 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.935386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.935869 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.935870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.936302 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.936303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.936351 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.936795 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.936800 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.938030 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.939369 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.939370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.939370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.939529 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.939707 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.939707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.939708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.939757 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.939968 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.939968 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.940003 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.940054 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.940137 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 11:38:13.940277 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.934338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35830 (local address 127.5.122.188:40581)
0620 11:38:13.934388 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.934390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.934398 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:13.934439 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.934439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.934449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.934491 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.934752 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.935302 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.935972 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.936206 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.936303 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.938126 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.938127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.938127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.938133 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.939292 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.939595 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.939596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.939596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.939643 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.939811 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.939811 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.939811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.939909 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.939958 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.940079 (+ 121us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.940183 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":19}
I20260620 11:38:13.941780 12420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Bootstrap starting.
I20260620 11:38:13.942695 12420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:13.943017 12420 log.cc:824] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:13.943830 12420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: No bootstrap required, opened a new log
I20260620 11:38:13.945317 12420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } }
I20260620 11:38:13.945456 12420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:13.945479 12420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bb8b2a0560640c092885231201d3923, State: Initialized, Role: FOLLOWER
I20260620 11:38:13.945637 12420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } }
I20260620 11:38:13.946030 12420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:13.945940 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } } }
I20260620 11:38:13.946123 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 11:38:13.947790 12436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:38:13.947853 12436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:38:13.950584 12352 master_runner.cc:428] Detected that this master 1bb8b2a0560640c092885231201d3923 is joining an existing cluster
I20260620 11:38:13.950666 12352 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.122.188:40581
I20260620 11:38:13.959657 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.952212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.0.0.1:35838)
0620 11:38:13.952707 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.952717 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.952785 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.953077 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.953078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.953246 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.953298 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.953299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.954141 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.954142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.955195 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.955197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.955702 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.955765 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.956336 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.956341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.956743 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.958493 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.958493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.958494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.958668 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.958894 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.958895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.958896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.958954 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.959222 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.959223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.959276 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.959371 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.959526 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":5,"thread_start_us":62,"threads_started":1}
I20260620 11:38:13.959659 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.952236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.5.122.188:40581)
0620 11:38:13.952270 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.952272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.952764 (+ 492us) server_negotiation.cc:407] Connection header received
0620 11:38:13.952845 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.952846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.952855 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.952883 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.953381 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.954057 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955306 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.955591 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955630 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.956869 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.956871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.956871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.956877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.958397 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.958761 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.958762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.958762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.958822 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.959032 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.959033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.959033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.959150 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.959232 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.959394 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.959579 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":19}
I20260620 11:38:13.978375 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.952021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57916 (local address 127.5.122.190:38037)
0620 11:38:13.952110 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.952113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.952410 (+ 297us) server_negotiation.cc:407] Connection header received
0620 11:38:13.952442 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.952443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.952456 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.952526 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.953070 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.953912 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955313 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.955615 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955654 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.975244 (+ 19590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.975247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.975247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.975255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.976929 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977410 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.977412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.977412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.977475 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977697 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.977699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.977699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.977870 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.977945 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.978170 (+ 225us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.978268 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 11:38:13.978438 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.952051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.5.122.189:33391)
0620 11:38:13.952158 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.952160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.952297 (+ 137us) server_negotiation.cc:407] Connection header received
0620 11:38:13.952411 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.952413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.952427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.952457 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.953110 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.953885 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955154 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.955458 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955504 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.975571 (+ 20067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.975573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.975573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.975590 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.977145 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977576 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.977578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.977578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.977640 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977844 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.977845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.977845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.978018 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.978079 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.978258 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.978342 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260620 11:38:13.978397 12442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.952100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:57916)
0620 11:38:13.952323 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.952331 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.952382 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.952544 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.952545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.952886 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.952977 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.952980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.954272 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.954272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.955205 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.955207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.955726 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955726 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.955779 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.974691 (+ 18912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.974698 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.975064 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.977039 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977041 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.977041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.977315 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.977557 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.977558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.977620 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.977951 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.977952 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.978056 (+ 104us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.978128 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.978276 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":54,"threads_started":1}
I20260620 11:38:13.978889 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.951986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.0.0.1:57650)
0620 11:38:13.952225 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.952238 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.952328 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.952548 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.952548 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.952936 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.953004 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.953007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.954026 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.954027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.955031 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.955034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.955604 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.955604 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.955704 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.974963 (+ 19259us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.974969 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.975394 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.977245 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977247 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.977247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.977469 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.977712 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.977713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.977713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.977770 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.978085 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.978086 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.978136 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.978558 (+ 422us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.978679 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":42,"thread_start_us":85,"threads_started":1}
I20260620 11:38:13.983006 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11762
Jun 20 11:38:13 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955492, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.190@KRBTEST.COM for oryx/127.5.122.188@KRBTEST.COM
I20260620 11:38:13.989197 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:36139
--local_ip_for_outbound_sockets=127.5.122.129
--tserver_master_addrs=127.5.122.190:38037
--webserver_port=42725
--webserver_interface=127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab
--principal=oryx/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:38:13.989356 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.981040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.0.0.1:35854)
0620 11:38:13.981100 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.981111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.981155 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.981298 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.981299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.981462 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:13.981510 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.981512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.982237 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.982238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.983025 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.983029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.984573 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.984575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.984652 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.985133 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:13.985139 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:13.986475 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:13.988171 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.988172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.988173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.988352 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.988581 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:13.988582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:13.988583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:13.988646 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:13.988980 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:13.988981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:13.989035 (+ 54us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.989101 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.989224 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 11:38:13.989790 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.981103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.5.122.188:40581)
0620 11:38:13.981148 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.981150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.981160 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:13.981229 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.981229 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.981239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.981294 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:13.981599 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.982154 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.983111 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.983412 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.983500 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.986580 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:13.986581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:13.986581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:13.986589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:13.988035 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.988436 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.988437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.988438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.988497 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:13.988718 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:13.988718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:13.988719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:13.988874 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:13.988945 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.989547 (+ 602us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.989650 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:38:13.990300 11700 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.122.188:40581: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1bb8b2a0560640c092885231201d3923"
member_type: NON_VOTER
last_known_addr {
host: "127.5.122.188"
port: 40581
}
attrs {
promote: true
}
}
dest_uuid: "587a77cc787b4b9196ffa4f87575a120"
cas_config_opid_index: 5
I20260620 11:38:13.990581 11700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } }
I20260620 11:38:13.991120 12259 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1bb8b2a0560640c092885231201d3923 (127.5.122.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:13.991250 12394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:13.992152 12394 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Refusing update from remote peer 587a77cc787b4b9196ffa4f87575a120: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 11:38:13.992147 12259 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.122.188:40581
I20260620 11:38:13.992228 12259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:13.992300 12259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:13.992666 12352 master.cc:537] Master@127.5.122.188:40581 shutting down...
I20260620 11:38:13.992683 12260 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 11:38:13.993729 11684 consensus_peers.cc:487] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 -> Peer 1bb8b2a0560640c092885231201d3923 (127.5.122.188:40581): Couldn't send request to peer 1bb8b2a0560640c092885231201d3923. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 11:38:13.995477 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.991092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.0.0.1:57656)
0620 11:38:13.991274 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:13.991285 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:13.991330 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:13.991560 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:13.991561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:13.991562 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:13.991611 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.991612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.992538 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.992539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.994188 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:13.994191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.995184 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.995186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:13.995272 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.995302 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 11:38:13.995365 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 11:38:13.995411 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260620 11:38:13.995878 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.991199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.5.122.189:33391)
0620 11:38:13.991246 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:13.991249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:13.991384 (+ 135us) server_negotiation.cc:407] Connection header received
0620 11:38:13.991413 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:13.991415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:13.991440 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:13.991496 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:13.991698 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.992436 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.994311 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:13.995082 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:13.995176 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:13.995273 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:13.995736 (+ 463us) server_negotiation.cc:299] Negotiation successful
0620 11:38:13.995792 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 11:38:13.996145 12352 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:13.996244 12352 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:13.996294 12352 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: stopping tablet replica
I20260620 11:38:13.996340 12308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 LEARNER]: Refusing update from remote peer 587a77cc787b4b9196ffa4f87575a120: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 11:38:13.996528 12260 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260620 11:38:13.997270 12308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1bb8b2a0560640c092885231201d3923 (127.5.122.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:13.998317 12337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587a77cc787b4b9196ffa4f87575a120. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:13.998416 12337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: LEARNER
I20260620 11:38:13.998461 12337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: NON_VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: true } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:13.998507 12337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: LEARNER
I20260620 11:38:13.998740 12260 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: attempting to promote NON_VOTER 207ee50512ab407abb46e19afe1e2291 to VOTER
I20260620 11:38:13.999010 12260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } }
I20260620 11:38:13.999646 12308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 LEARNER]: Refusing update from remote peer 587a77cc787b4b9196ffa4f87575a120: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 11:38:14.000039 12259 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 11:38:14.000562 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:13.999472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.0.0.1:35864)
0620 11:38:14.000478 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.000528 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:40581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":985}
I20260620 11:38:14.000764 12259 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 207ee50512ab407abb46e19afe1e2291 (127.5.122.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.001014 12308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 207ee50512ab407abb46e19afe1e2291 (127.5.122.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.001752 12259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.001837 12259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:14.001891 12259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.001938 12259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:14.002084 12337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.002177 12337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:38:14.002502 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.000835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.0.0.1:35880)
0620 11:38:14.002442 (+ 1607us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.002479 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:40581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1591}
W20260620 11:38:14.002580 11684 consensus_peers.cc:487] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 -> Peer 1bb8b2a0560640c092885231201d3923 (127.5.122.188:40581): Couldn't send request to peer 1bb8b2a0560640c092885231201d3923. Status: Network error: Client connection negotiation failed: client connection to 127.5.122.188:40581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 11:38:14.011507 12352 master.cc:552] Master@127.5.122.188:40581 shutdown complete.
I20260620 11:38:14.011574 12352 master_runner.cc:305] Clearing existing system tablet
I20260620 11:38:14.011754 12352 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:14.011818 12352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:14.012467 12352 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:14.013048 12450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.013172 12352 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:14.013222 12352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.013252 12352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
uuid: "1bb8b2a0560640c092885231201d3923"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "436edbe1af5efe648d9beafcb350eff2"
server_key_iv: "53138df11ecdcf8216cbe13abefffc14"
server_key_version: "encryptionkey@0"
I20260620 11:38:14.014017 12352 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:14.015458 12352 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 11:38:14.015548 12352 log.cc:1196] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 11:38:14.015784 12352 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Deleting consensus metadata
I20260620 11:38:14.015933 12352 master_runner.cc:315] Copying system tablet from 127.5.122.190:38037
I20260620 11:38:14.016569 12352 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: tablet copy: Beginning tablet copy session from remote peer at address 127.5.122.190:38037
I20260620 11:38:14.023483 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.016991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57928 (local address 127.5.122.190:38037)
0620 11:38:14.017047 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.017049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.017173 (+ 124us) server_negotiation.cc:407] Connection header received
0620 11:38:14.017257 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.017258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.017269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.017331 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.017675 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.018573 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.019353 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.019566 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.019665 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.020682 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.020683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.020684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.020691 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.022271 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.022644 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.022645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.022646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.022707 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.022911 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.022912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.022912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.023028 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.023102 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.023278 (+ 176us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.023364 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 11:38:14.023944 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.016939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:57928)
0620 11:38:14.017079 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.017089 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.017189 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.017338 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.017338 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.017527 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.017581 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.017583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.018681 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.018683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.019257 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.019259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.019667 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.019668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.019739 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.020245 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.020250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.020565 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.022373 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.022374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.022375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.022561 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.022777 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.022778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.022779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.022839 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.023116 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.023117 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.023164 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.023248 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.023364 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260620 11:38:14.024292 11720 tablet_copy_service.cc:140] P 587a77cc787b4b9196ffa4f87575a120: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1bb8b2a0560640c092885231201d3923 ({username='oryx', principal='oryx/127.5.122.188@KRBTEST.COM'} at 127.0.0.1:57928)
I20260620 11:38:14.024380 11720 tablet_copy_service.cc:161] P 587a77cc787b4b9196ffa4f87575a120: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1bb8b2a0560640c092885231201d3923 at {username='oryx', principal='oryx/127.5.122.188@KRBTEST.COM'} at 127.0.0.1:57928: session id = 1bb8b2a0560640c092885231201d3923-00000000000000000000000000000000
I20260620 11:38:14.024864 11720 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 11:38:14.025393 12352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 11:38:14.026518 12352 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: tablet copy: Starting download of 0 data blocks...
I20260620 11:38:14.026687 12352 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: tablet copy: Starting download of 1 WAL segments...
I20260620 11:38:14.027417 12352 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 11:38:14.028234 11720 tablet_copy_service.cc:342] P 587a77cc787b4b9196ffa4f87575a120: Request end of tablet copy session 1bb8b2a0560640c092885231201d3923-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.122.188@KRBTEST.COM'} at 127.0.0.1:57928
I20260620 11:38:14.028309 11720 tablet_copy_service.cc:433] P 587a77cc787b4b9196ffa4f87575a120: ending tablet copy session 1bb8b2a0560640c092885231201d3923-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1bb8b2a0560640c092885231201d3923
I20260620 11:38:14.030711 12352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:14.032099 12457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:14.032111 12460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:14.032363 12352 server_base.cc:1034] running on GCE node
W20260620 11:38:14.038141 12458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:14.038338 12352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:14.038520 12352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:14.039649 12352 hybrid_clock.cc:648] HybridClock initialized: now 1781955494039648 us; error 59 us; skew 500 ppm
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:14.041121 12352 init.cc:377] Logged in from keytab as oryx/127.5.122.188@KRBTEST.COM (short username oryx)
I20260620 11:38:14.041428 12352 webserver.cc:460] Webserver started at http://127.5.122.188:43613/ using document root <none> and password file <none>
I20260620 11:38:14.041550 12352 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:14.041586 12352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:14.042382 12352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:14.042922 12467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.043072 12352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 11:38:14.043118 12352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
uuid: "1bb8b2a0560640c092885231201d3923"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "436edbe1af5efe648d9beafcb350eff2"
server_key_iv: "53138df11ecdcf8216cbe13abefffc14"
server_key_version: "encryptionkey@0"
I20260620 11:38:14.043174 12352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:14.056738 12352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:14.058738 12352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:14.061895 12352 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.188:40581
I20260620 11:38:14.062288 12352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
I20260620 11:38:14.062605 12519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.188:40581 every 8 connection(s)
I20260620 11:38:14.063264 12520 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:38:14.063717 12520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Bootstrap starting.
I20260620 11:38:14.067904 12520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:38:14.068218 12520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: Bootstrap complete.
I20260620 11:38:14.068465 12520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } }
I20260620 11:38:14.068570 12520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1bb8b2a0560640c092885231201d3923, State: Initialized, Role: LEARNER
I20260620 11:38:14.068619 12520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } }
I20260620 11:38:14.068792 12520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:14.068785 12523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.069032 12523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: This master's current role is: LEARNER
I20260620 11:38:14.070873 12534 catalog_manager.cc:1259] Loaded cluster ID: 8feadcad4ee24a3cab7552f41b0d395d
I20260620 11:38:14.071089 12534 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: loading cluster ID for follower catalog manager: success
I20260620 11:38:14.071949 12534 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: acquiring CA information for follower catalog manager: success
I20260620 11:38:14.072259 12534 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 11:38:14.113317 12445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:14.113668 12445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:14.113719 12445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:14.115612 12445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:14.115690 12445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:14.115736 12445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:14.118180 12445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab
--principal=oryx/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:36139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=42725
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:14.118460 12445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:14.118734 12445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:14.124940 12540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:14.125073 12539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:14.124924 12542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:14.125361 12445 server_base.cc:1034] running on GCE node
I20260620 11:38:14.125586 12445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:14.125917 12445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:14.127125 12445 hybrid_clock.cc:648] HybridClock initialized: now 1781955494127112 us; error 30 us; skew 500 ppm
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:14.129693 12445 init.cc:377] Logged in from keytab as oryx/127.5.122.129@KRBTEST.COM (short username oryx)
I20260620 11:38:14.130199 12445 webserver.cc:460] Webserver started at http://127.5.122.129:42725/ using document root <none> and password file <none>
I20260620 11:38:14.130430 12445 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:14.130489 12445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:14.131898 12445 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:14.132550 12549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.132718 12445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:14.132791 12445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "2d80693a30fb469cb49af5d9122d0c73"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "5db2c6ab5eba92a6396b27aec25626d4"
server_key_iv: "632a9245971a6c50fe4fe0ac71a0681d"
server_key_version: "encryptionkey@0"
I20260620 11:38:14.133200 12445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:14.145401 12445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:14.147755 12445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:14.147918 12445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:14.148185 12445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:14.148511 12445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:14.148546 12445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.148569 12445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:14.148618 12445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.154654 12445 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:36139
I20260620 11:38:14.154755 12662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:36139 every 8 connection(s)
I20260620 11:38:14.155122 12445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:14.156010 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 12445
I20260620 11:38:14.156203 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11896
I20260620 11:38:14.163769 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:34363
--local_ip_for_outbound_sockets=127.5.122.130
--tserver_master_addrs=127.5.122.190:38037
--webserver_port=33063
--webserver_interface=127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab
--principal=oryx/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.188@KRBTEST.COM
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.189@KRBTEST.COM
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:14.168725 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.157606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:50913 (local address 127.5.122.189:33391)
0620 11:38:14.157697 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.157700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.157708 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:14.157733 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.157734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.157744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.157798 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.158511 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.159277 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161021 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.161323 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161367 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.165083 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.165086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.165086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.165096 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.166920 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.167491 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.167492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.167493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.167555 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.167864 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.167865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.167865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.168049 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.168099 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.168513 (+ 414us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.168604 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 11:38:14.169348 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.156363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:39623 (local address 127.5.122.188:40581)
0620 11:38:14.157355 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.157356 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.157363 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:38:14.157383 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.157384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.157394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.157546 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.158500 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.159525 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161007 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.161323 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161380 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.166029 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.166031 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.166031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.166040 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.167835 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.168215 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.168216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.168216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.168275 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.168461 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.168462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.168463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.168642 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.168702 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.169112 (+ 410us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.169232 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":59,"threads_started":1}
I20260620 11:38:14.169881 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.156235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.5.122.129:50913)
0620 11:38:14.157118 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.157129 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.157169 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.157848 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.157849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.158073 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.158383 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.158391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.159403 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.159403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.160898 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.160901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.161450 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.161604 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.163220 (+ 1616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.163225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.164929 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.167041 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.167049 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.167052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.167389 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.167635 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.167636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.167636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.167788 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.168149 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.168156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.168348 (+ 192us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.168871 (+ 523us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.169674 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":763,"thread_start_us":63,"threads_started":1}
I20260620 11:38:14.169881 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.155915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.5.122.129:39623)
0620 11:38:14.156140 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.156203 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.156955 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.157488 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.157489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.158046 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.158383 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.158391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.159610 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.159611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.160898 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.160901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.161443 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.161604 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.162475 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.162486 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.165628 (+ 3142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.167917 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.167917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.167918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.168141 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.168340 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.168341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.168341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.168394 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.168730 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.168731 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.168777 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.168858 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.169518 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":88,"spinlock_wait_cycles":7552,"thread_start_us":106,"threads_started":1}
I20260620 11:38:14.170460 12663 heartbeater.cc:347] Connected to a master server at 127.5.122.188:40581
I20260620 11:38:14.170483 12665 heartbeater.cc:347] Connected to a master server at 127.5.122.189:33391
I20260620 11:38:14.170562 12663 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.170563 12665 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.170714 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.156682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:52941 (local address 127.5.122.190:38037)
0620 11:38:14.156728 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.156731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.156744 (+ 13us) server_negotiation.cc:407] Connection header received
0620 11:38:14.157054 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.157055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.157068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.157130 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.158499 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.159338 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161004 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.161334 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161378 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.165719 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.165721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.165721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.165731 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.168366 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.168936 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.168938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.168938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.169002 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.169369 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.169370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.169371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.169524 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.169596 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.170496 (+ 900us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.170586 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 11:38:14.170804 12663 heartbeater.cc:511] Master 127.5.122.188:40581 requested a full tablet report, sending...
I20260620 11:38:14.170804 12665 heartbeater.cc:511] Master 127.5.122.189:33391 requested a full tablet report, sending...
I20260620 11:38:14.171361 12298 ts_manager.cc:194] Registered new tserver with Master: 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:14.171382 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.155915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:52941)
0620 11:38:14.156190 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.156204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.156930 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.157208 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.157212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.158044 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.158383 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.158391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.159441 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.159442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.160898 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.160901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.161461 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.161462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.161603 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.163791 (+ 2188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.163796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.165315 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.168553 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.168554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.168555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.168835 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.169118 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.169119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.169119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.169251 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.169779 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.169780 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.169834 (+ 54us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.171158 (+ 1324us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.171284 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":89,"thread_start_us":112,"threads_started":1}
I20260620 11:38:14.171368 12484 ts_manager.cc:194] Registered new tserver with Master: 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:14.171996 12664 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38037
I20260620 11:38:14.172070 12664 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.172147 12664 heartbeater.cc:511] Master 127.5.122.190:38037 requested a full tablet report, sending...
I20260620 11:38:14.172482 11700 ts_manager.cc:194] Re-registered known tserver with Master: 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:14.173044 11700 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:52941
W20260620 11:38:14.252606 12671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:14.252852 12671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:14.252879 12671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:14.254169 12671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:14.254222 12671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:14.254251 12671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:14.255826 12671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab
--principal=oryx/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:34363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=33063
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:14.256043 12671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:14.256239 12671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:14.261842 12679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:14.261878 12677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:14.261842 12676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:14.261894 12671 server_base.cc:1034] running on GCE node
I20260620 11:38:14.262339 12671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:14.262642 12671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:14.263816 12671 hybrid_clock.cc:648] HybridClock initialized: now 1781955494263810 us; error 38 us; skew 500 ppm
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:14.266429 12671 init.cc:377] Logged in from keytab as oryx/127.5.122.130@KRBTEST.COM (short username oryx)
I20260620 11:38:14.266959 12671 webserver.cc:460] Webserver started at http://127.5.122.130:33063/ using document root <none> and password file <none>
I20260620 11:38:14.267216 12671 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:14.267304 12671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:14.268710 12671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:14.269443 12686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.269608 12671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:38:14.269695 12671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "089d79306e8d4dfabd921e78de330db9"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "a4309938c9a0074b8a9b955130f53094"
server_key_iv: "eb07b207c52ef38844b75358f47e477d"
server_key_version: "encryptionkey@0"
I20260620 11:38:14.270224 12671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:14.286024 12671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:14.287979 12671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:14.288103 12671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:14.288292 12671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:14.288620 12671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:14.288656 12671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.288704 12671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:14.288738 12671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.295007 12671 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:34363
I20260620 11:38:14.295096 12799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:34363 every 8 connection(s)
I20260620 11:38:14.295459 12671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:14.300011 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 12671
I20260620 11:38:14.300180 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 12030
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.189@KRBTEST.COM
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.188@KRBTEST.COM
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:14.307921 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:40381
--local_ip_for_outbound_sockets=127.5.122.131
--tserver_master_addrs=127.5.122.190:38037
--webserver_port=36943
--webserver_interface=127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab
--principal=oryx/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:38:14.309123 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.296692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:37959 (local address 127.5.122.189:33391)
0620 11:38:14.296732 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.296734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.296743 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:14.297796 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.297798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.297813 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.297848 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.298822 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.299591 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.300878 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.301181 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.301227 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.305430 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.305431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.305432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.305439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.307127 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.307793 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.307794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.307795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.307938 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.308400 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.308401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.308401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.308530 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.308571 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.308934 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.309025 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:38:14.309122 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.296245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:58727 (local address 127.5.122.188:40581)
0620 11:38:14.296316 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.296318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.296548 (+ 230us) server_negotiation.cc:407] Connection header received
0620 11:38:14.297800 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.297802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.297817 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.297848 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.298824 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.299584 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.300878 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.301194 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.301230 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.305041 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.305043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.305044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.305053 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.307121 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.307787 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.307788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.307789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.307862 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.308316 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.308316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.308317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.308472 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.308515 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.308934 (+ 419us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.309025 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 11:38:14.309728 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.296234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:55581 (local address 127.5.122.190:38037)
0620 11:38:14.296289 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.296291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.296525 (+ 234us) server_negotiation.cc:407] Connection header received
0620 11:38:14.297788 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.297799 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.297812 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.297867 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.298823 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.299329 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.300876 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.301209 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.301253 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.306492 (+ 5239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.306494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.306495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.306503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.308343 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.308752 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.308753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.308753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.308809 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.309002 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.309003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.309003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.309141 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.309188 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.309506 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.309606 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:38:14.310017 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.296240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.5.122.130:58727)
0620 11:38:14.296400 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.296445 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.297667 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.298005 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.298010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.298490 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.298704 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.298710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.299689 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.299690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.300767 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.300769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.301330 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.301331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.301514 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.302418 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.302422 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.304847 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.307228 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.307239 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.307252 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.307689 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.308065 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.308066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.308066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.308226 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.308566 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.308573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.308762 (+ 189us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.309491 (+ 729us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.309714 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":66,"thread_start_us":103,"threads_started":1}
I20260620 11:38:14.310212 12805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.296247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:55581)
0620 11:38:14.296404 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.296445 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.297666 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.298018 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.298019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.298484 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.298704 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.298709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.299448 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.299449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.300767 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.300770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.301345 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.301346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.301514 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.302528 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.302534 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.305688 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.308446 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.308447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.308447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.308671 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.308879 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.308880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.308880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.308937 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.309318 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.309320 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.309373 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.309854 (+ 481us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.309996 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":13,"thread_start_us":89,"threads_started":1}
I20260620 11:38:14.310577 12800 heartbeater.cc:347] Connected to a master server at 127.5.122.188:40581
I20260620 11:38:14.310685 12800 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.310724 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.296544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.5.122.130:37959)
0620 11:38:14.296691 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.296701 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.297694 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.298013 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.298014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.298485 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.298704 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.298710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.299689 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.299690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.300767 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.300769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.301329 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.301330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.301514 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.302322 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.302331 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.305278 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.307228 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.307239 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.307242 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.307689 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.308264 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.308265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.308265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.308324 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.308613 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.308614 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.308764 (+ 150us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.309610 (+ 846us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.309951 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":126,"thread_start_us":69,"threads_started":1}
I20260620 11:38:14.310686 12801 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38037
I20260620 11:38:14.310813 12801 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.310920 12800 heartbeater.cc:511] Master 127.5.122.188:40581 requested a full tablet report, sending...
I20260620 11:38:14.310918 12801 heartbeater.cc:511] Master 127.5.122.190:38037 requested a full tablet report, sending...
I20260620 11:38:14.311143 12802 heartbeater.cc:347] Connected to a master server at 127.5.122.189:33391
I20260620 11:38:14.311178 12802 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.311237 12802 heartbeater.cc:511] Master 127.5.122.189:33391 requested a full tablet report, sending...
I20260620 11:38:14.311501 12484 ts_manager.cc:194] Registered new tserver with Master: 089d79306e8d4dfabd921e78de330db9 (127.5.122.130:34363)
I20260620 11:38:14.311532 12298 ts_manager.cc:194] Registered new tserver with Master: 089d79306e8d4dfabd921e78de330db9 (127.5.122.130:34363)
I20260620 11:38:14.311750 11700 ts_manager.cc:194] Re-registered known tserver with Master: 089d79306e8d4dfabd921e78de330db9 (127.5.122.130:34363)
I20260620 11:38:14.312286 11700 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:55581
W20260620 11:38:14.394757 12807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:14.395006 12807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:14.395035 12807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:14.396286 12807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:14.396333 12807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:14.396363 12807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:14.398140 12807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab
--principal=oryx/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:40381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=36943
--tserver_master_addrs=127.5.122.190:38037,127.5.122.189:33391,127.5.122.188:40581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:14.398379 12807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:14.398581 12807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:14.403961 12815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:14.403961 12813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:14.403986 12812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:14.404273 12807 server_base.cc:1034] running on GCE node
I20260620 11:38:14.404517 12807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:14.404791 12807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:14.406005 12807 hybrid_clock.cc:648] HybridClock initialized: now 1781955494405931 us; error 45 us; skew 500 ppm
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:14.408802 12807 init.cc:377] Logged in from keytab as oryx/127.5.122.131@KRBTEST.COM (short username oryx)
I20260620 11:38:14.409317 12807 webserver.cc:460] Webserver started at http://127.5.122.131:36943/ using document root <none> and password file <none>
I20260620 11:38:14.409584 12807 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:14.409643 12807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:14.411245 12807 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:14.411942 12822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.412204 12807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:14.412317 12807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "9d6bc15329f242cf8a865bb76548b697"
format_stamp: "Formatted at 2026-06-20 11:38:13 on dist-test-slave-szdr"
server_key: "1430241b61aa396b8d7986b2e2ea4013"
server_key_iv: "d7568b3a1813d03c696d1d2980694053"
server_key_version: "encryptionkey@0"
I20260620 11:38:14.412734 12807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:14.446103 12807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:14.448702 12807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:14.448891 12807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:14.449150 12807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:14.449481 12807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:14.449537 12807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.449574 12807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:14.449601 12807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:14.455715 12807 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:40381
I20260620 11:38:14.455763 12935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:40381 every 8 connection(s)
I20260620 11:38:14.456182 12807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.189@KRBTEST.COM
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.188@KRBTEST.COM
Jun 20 11:38:14 dist-test-slave-szdr krb5kdc[11655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955494, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:14.464723 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 12807
I20260620 11:38:14.468384 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.456981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:52197 (local address 127.5.122.190:38037)
0620 11:38:14.457029 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.457031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.457435 (+ 404us) server_negotiation.cc:407] Connection header received
0620 11:38:14.458519 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.458521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.458535 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.458566 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.459384 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.460154 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461283 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.461592 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461634 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.464889 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.464890 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.464891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.464899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.466637 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.467235 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.467237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.467237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.467306 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.467601 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.467602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.467602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.467738 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.467781 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.468165 (+ 384us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.468257 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:38:14.468958 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.457011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:52197)
0620 11:38:14.457219 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.457263 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.458392 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.458675 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.458680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.459060 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.459263 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.459269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.460241 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.460242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.461174 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.461176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.461715 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.461781 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.462420 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.462424 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.464761 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.466723 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.466730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.466732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.467130 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.467384 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.467385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.467385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.467520 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.467826 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.467832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.468010 (+ 178us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.468529 (+ 519us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.468708 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":35,"thread_start_us":103,"threads_started":1}
I20260620 11:38:14.469015 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.457512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:56953 (local address 127.5.122.188:40581)
0620 11:38:14.457583 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.457585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.457593 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:14.458518 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.458519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.458533 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.458566 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.459384 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.460138 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461278 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.461592 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461635 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.464299 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.464300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.464301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.464309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.466752 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.467484 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.467486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.467486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.467549 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.468030 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.468031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.468031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.468146 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.468186 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.468811 (+ 625us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.468903 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 11:38:14.469590 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.456987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:58303 (local address 127.5.122.189:33391)
0620 11:38:14.457039 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.457041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.457459 (+ 418us) server_negotiation.cc:407] Connection header received
0620 11:38:14.458514 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.458515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.458529 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.458591 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.459379 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.459876 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.460936 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.461151 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461242 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.463847 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.463849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.463849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.463857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.465985 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.467764 (+ 1779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.467765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.467765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.467833 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.468416 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.468417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.468417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.468568 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.468606 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.469419 (+ 813us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.469503 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:38:14.469406 12937 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38037
I20260620 11:38:14.470109 12937 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.470266 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.457010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.5.122.131:58303)
0620 11:38:14.457233 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.457265 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.458387 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.458678 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.458680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.459020 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.459263 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.459267 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.459996 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.459997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.460806 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.460808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.461251 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.461384 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.462003 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.462015 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.463736 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.466947 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.466947 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.466947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.467186 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.467906 (+ 720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.467906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.467907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.467959 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.468649 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.468650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.468687 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.469725 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.469857 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":30,"thread_start_us":138,"threads_started":1}
I20260620 11:38:14.470345 12937 heartbeater.cc:511] Master 127.5.122.190:38037 requested a full tablet report, sending...
I20260620 11:38:14.470126 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.457355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.5.122.131:56953)
0620 11:38:14.457486 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.457495 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.458404 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.458675 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.458680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.459061 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:14.459263 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.459269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.460241 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.460242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.461180 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.461181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.461714 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.461715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.461781 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.462332 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:14.462336 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:14.464200 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:14.466852 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.466853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.466853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.467395 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.467624 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:14.467625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:14.467625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:14.467684 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:14.468279 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:14.468280 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:14.468321 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.469122 (+ 801us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.469239 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":94,"thread_start_us":60,"threads_started":1}
I20260620 11:38:14.470870 11700 ts_manager.cc:194] Re-registered known tserver with Master: 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131:40381)
I20260620 11:38:14.471114 12936 heartbeater.cc:347] Connected to a master server at 127.5.122.188:40581
I20260620 11:38:14.471168 12936 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.471248 12936 heartbeater.cc:511] Master 127.5.122.188:40581 requested a full tablet report, sending...
I20260620 11:38:14.471400 11700 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:52197
I20260620 11:38:14.471632 12938 heartbeater.cc:347] Connected to a master server at 127.5.122.189:33391
I20260620 11:38:14.471688 12938 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:14.471755 12938 heartbeater.cc:511] Master 127.5.122.189:33391 requested a full tablet report, sending...
I20260620 11:38:14.471940 12484 ts_manager.cc:194] Registered new tserver with Master: 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131:40381)
I20260620 11:38:14.472579 12298 ts_manager.cc:194] Registered new tserver with Master: 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131:40381)
I20260620 11:38:14.474809 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.466054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.5.122.189:33391)
0620 11:38:14.466307 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.466309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.466440 (+ 131us) server_negotiation.cc:407] Connection header received
0620 11:38:14.466519 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.466520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.466532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.466619 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.467426 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.468282 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.469662 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.470014 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.470081 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.471761 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.471762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.471763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.471770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.473428 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.473790 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.473791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.473791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.473853 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.474105 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.474106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.474106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.474234 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.474349 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.474509 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.474593 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":57,"threads_started":1}
I20260620 11:38:14.477875 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.466155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.5.122.188:40581)
0620 11:38:14.466256 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.466258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.466852 (+ 594us) server_negotiation.cc:407] Connection header received
0620 11:38:14.466953 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.466954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.466967 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.467008 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.467763 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.468900 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.473148 (+ 4248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.473546 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.473588 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.475171 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.475172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.475172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.475179 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.476579 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.477030 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.477031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.477031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.477097 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.477304 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.477306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.477306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.477483 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.477580 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.477674 (+ 94us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.477757 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":53,"threads_started":1}
I20260620 11:38:14.479758 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.470713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.5.122.190:38037)
0620 11:38:14.470811 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.470813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.470827 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:38:14.470851 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.470852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.470862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.470897 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.472997 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.473726 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.475076 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.475371 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.475442 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.477037 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.477038 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.477038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.477046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.478761 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.479139 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.479140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.479140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.479191 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.479358 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.479359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.479359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.479450 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.479521 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.479608 (+ 87us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.479664 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 11:38:14.481339 11700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:14.483007 11700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:14.490801 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.486911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:34363 (local address 127.0.0.1:37220)
0620 11:38:14.487027 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.487040 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.487081 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.487481 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.487482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.487484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:14.487531 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.487533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.488908 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.488909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.489618 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.489620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.490554 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.490555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.490616 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.490619 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.490683 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.490725 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":3}
I20260620 11:38:14.490823 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.487083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37220 (local address 127.5.122.130:34363)
0620 11:38:14.487274 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.487279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.487290 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:38:14.487336 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.487338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.487357 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.487606 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:14.487669 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.488806 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.489727 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.490460 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.490538 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.490614 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.490698 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.490737 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":80,"threads_started":1}
I20260620 11:38:14.491154 12957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.486939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:40381 (local address 127.0.0.1:51222)
0620 11:38:14.487124 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.487135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.487186 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.487496 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.487497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.487499 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:14.487531 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.487533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.488851 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.488852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.489930 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.489932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.490929 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.490931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.490992 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.490996 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.491024 (+ 28us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.491066 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":36,"thread_start_us":92,"threads_started":1}
I20260620 11:38:14.491315 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.487217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.5.122.129:36139)
0620 11:38:14.487423 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.487428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.487535 (+ 107us) server_negotiation.cc:407] Connection header received
0620 11:38:14.487666 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.487668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.487683 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.487792 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:14.487905 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.488603 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.490280 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.490985 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.491060 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.491136 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.491205 (+ 69us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.491240 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260620 11:38:14.491312 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.486991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:36139 (local address 127.0.0.1:55776)
0620 11:38:14.487474 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.487484 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.487561 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.487788 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.487789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.487791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:14.487846 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.487848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.488703 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.488704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.490195 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.490197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.491074 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.491075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.491136 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.491141 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.491193 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.491237 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":95,"threads_started":1}
I20260620 11:38:14.491564 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.487169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51222 (local address 127.5.122.131:40381)
0620 11:38:14.487338 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.487342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.487351 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:14.487386 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.487388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.487403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.487605 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:14.487660 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.488760 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.490077 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.490806 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.491315 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.491411 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.491437 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.491480 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260620 11:38:14.491961 12734 tablet_service.cc:1467] Processing CreateTablet for tablet 9aedc908818a4e4bbe1d7c25a71a1464 (DEFAULT_TABLE table=test-table [id=820917296d3b43f58775c2e55cf9c5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:14.492039 12597 tablet_service.cc:1467] Processing CreateTablet for tablet 9aedc908818a4e4bbe1d7c25a71a1464 (DEFAULT_TABLE table=test-table [id=820917296d3b43f58775c2e55cf9c5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:14.492233 12870 tablet_service.cc:1467] Processing CreateTablet for tablet 9aedc908818a4e4bbe1d7c25a71a1464 (DEFAULT_TABLE table=test-table [id=820917296d3b43f58775c2e55cf9c5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:14.492321 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aedc908818a4e4bbe1d7c25a71a1464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:14.492321 12597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aedc908818a4e4bbe1d7c25a71a1464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:14.492482 12870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9aedc908818a4e4bbe1d7c25a71a1464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:14.495580 12961 tablet_bootstrap.cc:492] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Bootstrap starting.
I20260620 11:38:14.496223 12961 tablet_bootstrap.cc:654] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:14.496323 12963 tablet_bootstrap.cc:492] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Bootstrap starting.
I20260620 11:38:14.496806 12961 log.cc:824] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:14.496990 12963 tablet_bootstrap.cc:654] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:14.497259 12963 log.cc:824] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:14.497661 12961 tablet_bootstrap.cc:492] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: No bootstrap required, opened a new log
I20260620 11:38:14.497639 12962 tablet_bootstrap.cc:492] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Bootstrap starting.
I20260620 11:38:14.497723 12961 ts_tablet_manager.cc:1397] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:38:14.498157 12963 tablet_bootstrap.cc:492] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: No bootstrap required, opened a new log
I20260620 11:38:14.498229 12963 ts_tablet_manager.cc:1397] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:38:14.498291 12962 tablet_bootstrap.cc:654] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:14.498610 12962 log.cc:824] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:14.499423 12962 tablet_bootstrap.cc:492] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: No bootstrap required, opened a new log
I20260620 11:38:14.499504 12962 ts_tablet_manager.cc:1397] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:38:14.500068 12963 raft_consensus.cc:348] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.500240 12963 raft_consensus.cc:374] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:14.500285 12963 raft_consensus.cc:729] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d80693a30fb469cb49af5d9122d0c73, State: Initialized, Role: FOLLOWER
I20260620 11:38:14.500418 12963 consensus_queue.cc:260] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.500509 12961 raft_consensus.cc:348] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.500674 12961 raft_consensus.cc:374] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:14.500717 12961 raft_consensus.cc:729] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6bc15329f242cf8a865bb76548b697, State: Initialized, Role: FOLLOWER
I20260620 11:38:14.500717 12963 ts_tablet_manager.cc:1428] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:38:14.500864 12664 heartbeater.cc:503] Master 127.5.122.190:38037 was elected leader, sending a full tablet report...
I20260620 11:38:14.500957 12961 consensus_queue.cc:260] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.501984 12961 ts_tablet_manager.cc:1428] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 11:38:14.502130 12937 heartbeater.cc:503] Master 127.5.122.190:38037 was elected leader, sending a full tablet report...
I20260620 11:38:14.502774 12962 raft_consensus.cc:348] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.502947 12962 raft_consensus.cc:374] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:14.502987 12962 raft_consensus.cc:729] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089d79306e8d4dfabd921e78de330db9, State: Initialized, Role: FOLLOWER
I20260620 11:38:14.503115 12962 consensus_queue.cc:260] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.503384 12962 ts_tablet_manager.cc:1428] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 11:38:14.503536 12801 heartbeater.cc:503] Master 127.5.122.190:38037 was elected leader, sending a full tablet report...
W20260620 11:38:14.546375 12803 tablet.cc:2369] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:38:14.571744 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.568517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.0.0.1:35900)
0620 11:38:14.568636 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.568647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.568726 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.568934 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.568935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.568937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:14.568994 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.568997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.569885 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.569886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.570948 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.570952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.571540 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.571541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.571588 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.571591 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.571638 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.571677 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260620 11:38:14.571744 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.568635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.5.122.188:40581)
0620 11:38:14.568703 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.568705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.568716 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:38:14.568801 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.568802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.568834 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.568907 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:14.569078 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.569800 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.571032 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.571448 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.571513 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.571574 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.571658 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.571689 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:38:14.572151 12494 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260620 11:38:14.573149 12970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 587a77cc787b4b9196ffa4f87575a120. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.573267 12970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: This master's current role is: LEARNER
W20260620 11:38:14.657744 12666 tablet.cc:2369] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:38:14.707029 12939 tablet.cc:2369] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:38:14.818284 12969 raft_consensus.cc:482] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:14.818432 12969 raft_consensus.cc:504] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.818934 12969 leader_election.cc:288] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131:40381), 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:14.822108 12976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.819160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:40381 (local address 127.5.122.130:59065)
0620 11:38:14.819305 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.819315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.819415 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.819569 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.819569 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.819572 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:14.819668 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.819670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.820385 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.820386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.821199 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.821201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.821770 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.821771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.821845 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.821850 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.821936 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.822004 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":26,"thread_start_us":70,"threads_started":1}
I20260620 11:38:14.822098 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.819218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:59065 (local address 127.5.122.131:40381)
0620 11:38:14.819274 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.819276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.819422 (+ 146us) server_negotiation.cc:407] Connection header received
0620 11:38:14.819479 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.819480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.819490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.819553 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:14.819771 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.820261 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.821311 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.821684 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.821741 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.821749 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.821943 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.822029 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:38:14.822225 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.819237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:47041 (local address 127.5.122.129:36139)
0620 11:38:14.819287 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.819290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.819393 (+ 103us) server_negotiation.cc:407] Connection header received
0620 11:38:14.819466 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.819467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.819478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.819560 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:14.819734 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.820239 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.821318 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.821696 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.821788 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.821797 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.822063 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.822124 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:38:14.822476 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.819151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:36139 (local address 127.5.122.130:47041)
0620 11:38:14.819297 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:14.819309 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:14.819379 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:14.819558 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:14.819559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:14.819561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:14.819657 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.819659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.820334 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.820335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.821196 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:14.821198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.821812 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.821813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:14.821932 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.821937 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:38:14.822321 (+ 384us) client_negotiation.cc:240] Negotiation successful
0620 11:38:14.822383 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":31,"thread_start_us":72,"threads_started":1}
I20260620 11:38:14.822486 12890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aedc908818a4e4bbe1d7c25a71a1464" candidate_uuid: "089d79306e8d4dfabd921e78de330db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6bc15329f242cf8a865bb76548b697" is_pre_election: true
I20260620 11:38:14.822705 12890 raft_consensus.cc:2454] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089d79306e8d4dfabd921e78de330db9 in term 0.
I20260620 11:38:14.822908 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aedc908818a4e4bbe1d7c25a71a1464" candidate_uuid: "089d79306e8d4dfabd921e78de330db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d80693a30fb469cb49af5d9122d0c73" is_pre_election: true
I20260620 11:38:14.822968 12690 leader_election.cc:302] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089d79306e8d4dfabd921e78de330db9, 9d6bc15329f242cf8a865bb76548b697; no voters:
I20260620 11:38:14.823101 12617 raft_consensus.cc:2454] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 089d79306e8d4dfabd921e78de330db9 in term 0.
I20260620 11:38:14.823132 12969 raft_consensus.cc:2781] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:14.823199 12969 raft_consensus.cc:482] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:14.823231 12969 raft_consensus.cc:3037] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:14.824007 12969 raft_consensus.cc:504] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.824174 12969 leader_election.cc:288] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [CANDIDATE]: Term 1 election: Requested vote from peers 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131:40381), 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:14.824323 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aedc908818a4e4bbe1d7c25a71a1464" candidate_uuid: "089d79306e8d4dfabd921e78de330db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d80693a30fb469cb49af5d9122d0c73"
I20260620 11:38:14.824395 12617 raft_consensus.cc:3037] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:14.825049 12617 raft_consensus.cc:2454] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089d79306e8d4dfabd921e78de330db9 in term 1.
I20260620 11:38:14.825183 12890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aedc908818a4e4bbe1d7c25a71a1464" candidate_uuid: "089d79306e8d4dfabd921e78de330db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6bc15329f242cf8a865bb76548b697"
I20260620 11:38:14.825224 12688 leader_election.cc:302] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 089d79306e8d4dfabd921e78de330db9, 2d80693a30fb469cb49af5d9122d0c73; no voters:
I20260620 11:38:14.825275 12890 raft_consensus.cc:3037] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:14.825389 12969 raft_consensus.cc:2781] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:14.825534 12969 raft_consensus.cc:686] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 1 LEADER]: Becoming Leader. State: Replica: 089d79306e8d4dfabd921e78de330db9, State: Running, Role: LEADER
I20260620 11:38:14.825631 12969 consensus_queue.cc:237] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:14.826279 12890 raft_consensus.cc:2454] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 089d79306e8d4dfabd921e78de330db9 in term 1.
I20260620 11:38:14.826751 11698 catalog_manager.cc:5495] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 089d79306e8d4dfabd921e78de330db9 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "089d79306e8d4dfabd921e78de330db9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } health_report { overall_health: UNKNOWN } } }
I20260620 11:38:14.827441 12260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: NON_VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: true } } }
I20260620 11:38:14.827571 12260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:14.828178 12448 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: attempting to promote NON_VOTER 1bb8b2a0560640c092885231201d3923 to VOTER
I20260620 11:38:14.828305 12448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:14.828953 12494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 LEARNER]: Refusing update from remote peer 587a77cc787b4b9196ffa4f87575a120: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260620 11:38:14.828984 12308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Refusing update from remote peer 587a77cc787b4b9196ffa4f87575a120: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260620 11:38:14.829178 12260 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260620 11:38:14.829284 12979 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260620 11:38:14.829912 12448 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1bb8b2a0560640c092885231201d3923 (127.5.122.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.830072 12308 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1bb8b2a0560640c092885231201d3923 (127.5.122.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.830946 12448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.831036 12448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:14.830989 12260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.831089 12260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:14.831108 12448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.831162 12448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:14.831171 12981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.831260 12981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:38:14.831241 12494 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1bb8b2a0560640c092885231201d3923 (127.5.122.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.832149 12970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "587a77cc787b4b9196ffa4f87575a120" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:14.832276 12970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:38:14.843912 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.841200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.5.122.130:34363)
0620 11:38:14.841262 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.841264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.841273 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:14.841328 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.841329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.841346 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.841409 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:14.841595 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.842168 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.843002 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.843192 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.843276 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.843474 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:14.843596 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:14.843676 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.843732 (+ 56us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.843756 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 11:38:14.845914 12617 raft_consensus.cc:1261] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 1 FOLLOWER]: Refusing update from remote peer 089d79306e8d4dfabd921e78de330db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:14.845914 12890 raft_consensus.cc:1261] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 1 FOLLOWER]: Refusing update from remote peer 089d79306e8d4dfabd921e78de330db9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:14.846362 12977 consensus_queue.cc:1035] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:14.846506 12969 consensus_queue.cc:1035] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:14.850199 12983 mvcc.cc:205] Tried to move back new op lower bound from 7298889706885476352 to 7298889706806353920. Current Snapshot: MvccSnapshot[applied={T|T < 7298889706885476352 or (T in {7298889706885476352})}]
I20260620 11:38:14.850347 12985 mvcc.cc:205] Tried to move back new op lower bound from 7298889706885476352 to 7298889706806353920. Current Snapshot: MvccSnapshot[applied={T|T < 7298889706885476352}]
I20260620 11:38:14.851186 12984 mvcc.cc:205] Tried to move back new op lower bound from 7298889706885476352 to 7298889706806353920. Current Snapshot: MvccSnapshot[applied={T|T < 7298889706885476352}]
I20260620 11:38:14.852106 11700 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:14.852200 11700 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:14.853415 11700 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 587a77cc787b4b9196ffa4f87575a120: Sending DeleteTablet for 3 replicas of tablet 9aedc908818a4e4bbe1d7c25a71a1464
I20260620 11:38:14.853739 12597 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aedc908818a4e4bbe1d7c25a71a1464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:14 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:55776
I20260620 11:38:14.853741 12870 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aedc908818a4e4bbe1d7c25a71a1464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:14 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:51222
I20260620 11:38:14.854156 12734 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aedc908818a4e4bbe1d7c25a71a1464 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:14 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37220
I20260620 11:38:14.854364 12996 tablet_replica.cc:330] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: stopping tablet replica
I20260620 11:38:14.854444 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 11668
I20260620 11:38:14.854462 12996 raft_consensus.cc:2229] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:38:14.855499 12996 raft_consensus.cc:2258] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:14.855669 12993 tablet_replica.cc:330] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: stopping tablet replica
I20260620 11:38:14.855777 12993 raft_consensus.cc:2229] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:14.855887 12993 raft_consensus.cc:2258] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:14.855997 12996 ts_tablet_manager.cc:1905] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:14.856419 12993 ts_tablet_manager.cc:1905] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:14.857168 12994 tablet_replica.cc:330] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: stopping tablet replica
I20260620 11:38:14.857280 12994 raft_consensus.cc:2229] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:14.857398 12994 raft_consensus.cc:2258] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:14.858209 12993 ts_tablet_manager.cc:1918] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:14.858294 12993 log.cc:1196] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/9aedc908818a4e4bbe1d7c25a71a1464
I20260620 11:38:14.858403 12996 ts_tablet_manager.cc:1918] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:14.858522 12996 log.cc:1196] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/9aedc908818a4e4bbe1d7c25a71a1464
I20260620 11:38:14.858592 12993 ts_tablet_manager.cc:1939] T 9aedc908818a4e4bbe1d7c25a71a1464 P 2d80693a30fb469cb49af5d9122d0c73: Deleting consensus metadata
I20260620 11:38:14.858876 12996 ts_tablet_manager.cc:1939] T 9aedc908818a4e4bbe1d7c25a71a1464 P 089d79306e8d4dfabd921e78de330db9: Deleting consensus metadata
I20260620 11:38:14.859398 12994 ts_tablet_manager.cc:1905] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 11:38:14.860630 12552 connection.cc:535] server connection from 127.0.0.1:55776 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260620 11:38:14.860813 12689 connection.cc:535] server connection from 127.0.0.1:37220 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 11:38:14.862450 12994 ts_tablet_manager.cc:1918] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:14.862531 12994 log.cc:1196] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/9aedc908818a4e4bbe1d7c25a71a1464
I20260620 11:38:14.862828 12994 ts_tablet_manager.cc:1939] T 9aedc908818a4e4bbe1d7c25a71a1464 P 9d6bc15329f242cf8a865bb76548b697: Deleting consensus metadata
W20260620 11:38:14.863231 12825 connection.cc:441] server connection from 127.0.0.1:51222 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51222 (request call id 1) could send its response
I20260620 11:38:14.867343 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.854381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.5.122.188:40581)
0620 11:38:14.854421 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.854424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.856115 (+ 1691us) server_negotiation.cc:407] Connection header received
0620 11:38:14.856225 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.856226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.856238 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.856269 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.857560 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.858373 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.861621 (+ 3248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.861927 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.862045 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.863912 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.863914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.863914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.863922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.865860 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.866284 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.866286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.866286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.866345 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.866554 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.866555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.866555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.866674 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.866747 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.867198 (+ 451us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.867277 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:38:14.871791 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.862578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35916 (local address 127.5.122.188:40581)
0620 11:38:14.862680 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.862682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.864415 (+ 1733us) server_negotiation.cc:407] Connection header received
0620 11:38:14.864519 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.864520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.864532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.864563 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.865246 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.866087 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.867134 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.867427 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.867495 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.868977 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.868979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.868979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.868987 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.870569 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.870973 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.870974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.870975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.871030 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.871251 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.871252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.871252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.871373 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.871479 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.871587 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.871666 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 11:38:14.872216 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:14.862190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57670 (local address 127.5.122.189:33391)
0620 11:38:14.862238 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:14.862240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:14.863037 (+ 797us) server_negotiation.cc:407] Connection header received
0620 11:38:14.863146 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:14.863147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:14.863164 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:14.863196 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:14.864336 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.865131 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.866290 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:14.866622 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:14.866668 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:14.868849 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:14.868851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:14.868851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:14.868860 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:14.870961 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.871360 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.871361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.871362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.871418 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:14.871622 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:14.871622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:14.871623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:14.871753 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:14.871823 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:14.872023 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 11:38:14.872120 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:38:15.850910 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:15.850503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:40615)
0620 11:38:15.850692 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:15.850802 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260620 11:38:15.850984 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:15.850711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:60003)
0620 11:38:15.850850 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:15.850930 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
W20260620 11:38:15.851209 12937 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:38037 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
W20260620 11:38:15.851655 12664 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:38037 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:15.853724 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:15.853390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:40297)
0620 11:38:15.853613 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:15.853682 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":56,"threads_started":1}
W20260620 11:38:15.854030 12801 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:38037 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:16.353715 13007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 587a77cc787b4b9196ffa4f87575a120)
I20260620 11:38:16.353856 13007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:16.354197 13007 leader_election.cc:288] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037), 207ee50512ab407abb46e19afe1e2291 (127.5.122.189:33391)
I20260620 11:38:16.354419 13008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 587a77cc787b4b9196ffa4f87575a120)
I20260620 11:38:16.354538 13008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:16.354593 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.354368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45752)
0620 11:38:16.354486 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.354542 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":49,"threads_started":1}
W20260620 11:38:16.354723 12471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:16.354848 13008 leader_election.cc:288] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037), 1bb8b2a0560640c092885231201d3923 (127.5.122.188:40581)
I20260620 11:38:16.355125 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.354922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45762)
0620 11:38:16.355022 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.355082 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
I20260620 11:38:16.355278 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.355104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45764)
0620 11:38:16.355220 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.355246 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
W20260620 11:38:16.355321 12285 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
W20260620 11:38:16.355366 12471 leader_election.cc:334] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037): Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:16.355891 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.355733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45772)
0620 11:38:16.355842 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.355866 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
W20260620 11:38:16.355968 12285 leader_election.cc:334] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037): Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:16.358013 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.354459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.5.122.189:33391)
0620 11:38:16.354639 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:16.354733 (+ 94us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:16.354745 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:38:16.354806 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:16.354807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:16.354998 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:16.355246 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:16.355285 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.356114 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.357266 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.357652 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.357734 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:16.357746 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:16.357886 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 11:38:16.357926 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":44,"threads_started":1}
I20260620 11:38:16.358031 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.354382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:33391 (local address 127.0.0.1:43262)
0620 11:38:16.354566 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.354579 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:16.354667 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:16.355100 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:16.355101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:16.355104 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:16.355175 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:16.355177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.356198 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.356199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:16.357179 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:16.357182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.357746 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.357747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:16.357813 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:16.357816 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:16.357881 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 11:38:16.357928 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":38,"threads_started":1}
I20260620 11:38:16.358264 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.354981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:40581 (local address 127.0.0.1:51150)
0620 11:38:16.355273 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.355284 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:16.355339 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:16.355625 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:16.355626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:16.355628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:16.355665 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:16.355666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.356434 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.356435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:16.357442 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:16.357445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.358014 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.358015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:16.358082 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:16.358087 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:38:16.358144 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 11:38:16.358197 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":43,"threads_started":1}
I20260620 11:38:16.358273 13016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.355165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.5.122.188:40581)
0620 11:38:16.355482 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:16.355484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:16.355493 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:16.355516 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:16.355517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:16.355527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:16.355601 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:16.355742 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.356343 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.357528 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:16.357900 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:16.358016 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:16.358026 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:16.358173 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 11:38:16.358223 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":66,"threads_started":1}
I20260620 11:38:16.358263 12308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1bb8b2a0560640c092885231201d3923" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "207ee50512ab407abb46e19afe1e2291" is_pre_election: true
I20260620 11:38:16.358415 12308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb8b2a0560640c092885231201d3923 in term 1.
I20260620 11:38:16.358487 12494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "207ee50512ab407abb46e19afe1e2291" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb8b2a0560640c092885231201d3923" is_pre_election: true
I20260620 11:38:16.358589 12494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 207ee50512ab407abb46e19afe1e2291 in term 1.
I20260620 11:38:16.358588 12468 leader_election.cc:302] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bb8b2a0560640c092885231201d3923, 207ee50512ab407abb46e19afe1e2291; no voters: 587a77cc787b4b9196ffa4f87575a120
I20260620 11:38:16.358723 13007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 11:38:16.358753 12282 leader_election.cc:302] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bb8b2a0560640c092885231201d3923, 207ee50512ab407abb46e19afe1e2291; no voters: 587a77cc787b4b9196ffa4f87575a120
I20260620 11:38:16.358780 13007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 587a77cc787b4b9196ffa4f87575a120)
I20260620 11:38:16.358804 13007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 1 FOLLOWER]: Advancing to term 2
I20260620 11:38:16.358871 13008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 11:38:16.358918 13008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 587a77cc787b4b9196ffa4f87575a120)
I20260620 11:38:16.358946 13008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 1 FOLLOWER]: Advancing to term 2
I20260620 11:38:16.359516 13007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:16.359673 13007 leader_election.cc:288] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [CANDIDATE]: Term 2 election: Requested vote from peers 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037), 207ee50512ab407abb46e19afe1e2291 (127.5.122.189:33391)
I20260620 11:38:16.359663 13008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:16.359822 13008 leader_election.cc:288] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 2 election: Requested vote from peers 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037), 1bb8b2a0560640c092885231201d3923 (127.5.122.188:40581)
I20260620 11:38:16.359875 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.359755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45792)
0620 11:38:16.359813 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.359843 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 11:38:16.359941 12308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1bb8b2a0560640c092885231201d3923" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "207ee50512ab407abb46e19afe1e2291"
I20260620 11:38:16.360003 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.359879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45808)
0620 11:38:16.359952 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.359977 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 11:38:16.359987 12494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "207ee50512ab407abb46e19afe1e2291" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb8b2a0560640c092885231201d3923"
I20260620 11:38:16.360018 12308 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1bb8b2a0560640c092885231201d3923 in current term 2: Already voted for candidate 207ee50512ab407abb46e19afe1e2291 in this term.
I20260620 11:38:16.360057 12494 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 207ee50512ab407abb46e19afe1e2291 in current term 2: Already voted for candidate 1bb8b2a0560640c092885231201d3923 in this term.
I20260620 11:38:16.360282 13014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.360214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45824)
0620 11:38:16.360243 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.360263 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260620 11:38:16.360327 13010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.360195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:45820)
0620 11:38:16.360279 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.360310 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260620 11:38:16.360404 12285 leader_election.cc:334] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037): Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
W20260620 11:38:16.360409 12471 leader_election.cc:334] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037): Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:16.360452 12471 leader_election.cc:302] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1bb8b2a0560640c092885231201d3923; no voters: 207ee50512ab407abb46e19afe1e2291, 587a77cc787b4b9196ffa4f87575a120
I20260620 11:38:16.360452 12285 leader_election.cc:302] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 207ee50512ab407abb46e19afe1e2291; no voters: 1bb8b2a0560640c092885231201d3923, 587a77cc787b4b9196ffa4f87575a120
I20260620 11:38:16.360529 13008 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 11:38:16.360571 13007 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 11:38:16.851900 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.851573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:45609)
0620 11:38:16.851715 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.851761 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260620 11:38:16.853049 13020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.852197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:60179)
0620 11:38:16.852950 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.852993 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":71,"threads_started":1}
I20260620 11:38:16.854549 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:16.854378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:57113)
0620 11:38:16.854469 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:16.854513 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":53,"threads_started":1}
I20260620 11:38:17.853048 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:17.852813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:38807)
0620 11:38:17.852945 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:17.852985 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260620 11:38:17.853745 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:17.853461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:47879)
0620 11:38:17.853628 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:17.853695 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260620 11:38:17.855122 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:17.854931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:57087)
0620 11:38:17.855051 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:17.855086 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":50,"threads_started":1}
I20260620 11:38:18.009804 13025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:18.009892 13025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:18.010125 13025 leader_election.cc:288] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037), 1bb8b2a0560640c092885231201d3923 (127.5.122.188:40581)
I20260620 11:38:18.010449 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.010243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:46048)
0620 11:38:18.010357 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.010404 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260620 11:38:18.010391 12494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "207ee50512ab407abb46e19afe1e2291" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb8b2a0560640c092885231201d3923" is_pre_election: true
I20260620 11:38:18.010504 12494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 207ee50512ab407abb46e19afe1e2291 in term 2.
I20260620 11:38:18.010684 12282 leader_election.cc:302] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb8b2a0560640c092885231201d3923, 207ee50512ab407abb46e19afe1e2291; no voters:
I20260620 11:38:18.010802 13025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260620 11:38:18.010859 13025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:18.010879 13025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 2 FOLLOWER]: Advancing to term 3
I20260620 11:38:18.011037 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.010913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:46058)
0620 11:38:18.010950 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.011003 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260620 11:38:18.011121 12285 leader_election.cc:334] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037): Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:18.011727 13025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:18.011880 13025 leader_election.cc:288] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 3 election: Requested vote from peers 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037), 1bb8b2a0560640c092885231201d3923 (127.5.122.188:40581)
I20260620 11:38:18.012053 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.011959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:46068)
0620 11:38:18.012000 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.012032 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 11:38:18.012032 12494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "207ee50512ab407abb46e19afe1e2291" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1bb8b2a0560640c092885231201d3923"
I20260620 11:38:18.012151 12494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 2 FOLLOWER]: Advancing to term 3
I20260620 11:38:18.012439 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.012322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:46074)
0620 11:38:18.012389 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.012419 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 11:38:18.012534 12285 leader_election.cc:334] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037): Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
I20260620 11:38:18.012912 12494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 207ee50512ab407abb46e19afe1e2291 in term 3.
I20260620 11:38:18.013073 12282 leader_election.cc:302] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1bb8b2a0560640c092885231201d3923, 207ee50512ab407abb46e19afe1e2291; no voters: 587a77cc787b4b9196ffa4f87575a120
I20260620 11:38:18.013167 13025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 11:38:18.013288 13025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [term 3 LEADER]: Becoming Leader. State: Replica: 207ee50512ab407abb46e19afe1e2291, State: Running, Role: LEADER
I20260620 11:38:18.013381 13025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } }
I20260620 11:38:18.013734 13028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 207ee50512ab407abb46e19afe1e2291. Latest consensus state: current_term: 3 leader_uuid: "207ee50512ab407abb46e19afe1e2291" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:18.013831 13028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:18.014043 13030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:18.014437 13030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:18.014536 13030 catalog_manager.cc:1259] Loaded cluster ID: 8feadcad4ee24a3cab7552f41b0d395d
I20260620 11:38:18.014573 13030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:18.014758 13030 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:18.014842 13030 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Loaded TSK: 0
I20260620 11:38:18.015022 13030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:38:18.042590 12297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:18.044416 12297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:18.045614 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.045481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:46086)
0620 11:38:18.045541 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.045579 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 11:38:18.045668 12494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [term 3 FOLLOWER]: Refusing update from remote peer 207ee50512ab407abb46e19afe1e2291: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260620 11:38:18.046036 13028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260620 11:38:18.046119 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.045937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.0.0.1:46098)
0620 11:38:18.045999 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.046027 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 11:38:18.046245 12285 consensus_peers.cc:487] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 -> Peer 587a77cc787b4b9196ffa4f87575a120 (127.5.122.190:38037): Couldn't send request to peer 587a77cc787b4b9196ffa4f87575a120. Status: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 11:38:18.046722 13032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 207ee50512ab407abb46e19afe1e2291. Latest consensus state: current_term: 3 leader_uuid: "207ee50512ab407abb46e19afe1e2291" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:18.046844 13032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:38:18.047385 13028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "207ee50512ab407abb46e19afe1e2291" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:18.047485 13028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:18.047712 13032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "207ee50512ab407abb46e19afe1e2291" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "587a77cc787b4b9196ffa4f87575a120" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38037 } } peers { permanent_uuid: "207ee50512ab407abb46e19afe1e2291" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 33391 } attrs { promote: false } } peers { permanent_uuid: "1bb8b2a0560640c092885231201d3923" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 40581 } attrs { promote: false } } }
I20260620 11:38:18.047807 13032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bb8b2a0560640c092885231201d3923 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:38:18.052783 13026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.049297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:34363 (local address 127.0.0.1:41932)
0620 11:38:18.049351 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.049363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:18.049414 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:18.049831 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:18.049832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:18.049834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:18.049886 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.049888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.050830 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.050831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.051862 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.051863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.052527 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.052528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.052595 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.052601 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:38:18.052648 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:38:18.052692 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 11:38:18.052892 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.049522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:36139 (local address 127.0.0.1:51098)
0620 11:38:18.049724 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.049735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:18.049786 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:18.050261 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:18.050262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:18.050264 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:18.050301 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.050302 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.051089 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.051090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.051886 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.051889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.052633 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.052634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.052688 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.052690 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:18.052740 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 11:38:18.052773 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260620 11:38:18.052899 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.049603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.5.122.129:36139)
0620 11:38:18.049740 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:18.049742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:18.049775 (+ 33us) server_negotiation.cc:407] Connection header received
0620 11:38:18.049999 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:18.050000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:18.050012 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:18.050221 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:18.050370 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.051003 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.051981 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.052547 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.052643 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.052654 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:18.052771 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 11:38:18.052821 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260620 11:38:18.053061 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.049487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.5.122.131:40381)
0620 11:38:18.049595 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:18.049597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:18.049605 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:18.049633 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:18.049634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:18.049659 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:18.049728 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:18.049905 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.050468 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.052008 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.052577 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.052641 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.052652 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:18.052927 (+ 275us) server_negotiation.cc:299] Negotiation successful
0620 11:38:18.052981 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260620 11:38:18.053088 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.049346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:40381 (local address 127.0.0.1:33062)
0620 11:38:18.049475 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.049486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:18.049536 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:18.049786 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:18.049786 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:18.049789 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:18.049844 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.049846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.050560 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.050562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.051699 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.051701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.052688 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.052689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.052757 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.052763 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:38:18.052811 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 11:38:18.052853 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260620 11:38:18.053143 12597 tablet_service.cc:1467] Processing CreateTablet for tablet 2d79507399cc4569bc42f63dc4c1b010 (DEFAULT_TABLE table=test-table [id=cb53f63ec63544ed93c84955902a8f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:18.053288 12597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d79507399cc4569bc42f63dc4c1b010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:18.053467 12870 tablet_service.cc:1467] Processing CreateTablet for tablet 2d79507399cc4569bc42f63dc4c1b010 (DEFAULT_TABLE table=test-table [id=cb53f63ec63544ed93c84955902a8f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:18.053586 12870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d79507399cc4569bc42f63dc4c1b010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:18.053634 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.049356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.5.122.130:34363)
0620 11:38:18.049665 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:18.049667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:18.049694 (+ 27us) server_negotiation.cc:407] Connection header received
0620 11:38:18.049724 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:18.049725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:18.049738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:18.049954 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:18.050015 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.050745 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.051941 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.052423 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.052512 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.052525 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:18.053532 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0620 11:38:18.053581 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":47,"threads_started":1}
I20260620 11:38:18.053855 12734 tablet_service.cc:1467] Processing CreateTablet for tablet 2d79507399cc4569bc42f63dc4c1b010 (DEFAULT_TABLE table=test-table [id=cb53f63ec63544ed93c84955902a8f55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:18.053954 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d79507399cc4569bc42f63dc4c1b010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:18.054894 13043 tablet_bootstrap.cc:492] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73: Bootstrap starting.
I20260620 11:38:18.055202 13045 tablet_bootstrap.cc:492] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: Bootstrap starting.
I20260620 11:38:18.055409 13043 tablet_bootstrap.cc:654] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:18.055428 13044 tablet_bootstrap.cc:492] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: Bootstrap starting.
I20260620 11:38:18.055717 13045 tablet_bootstrap.cc:654] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:18.055840 13044 tablet_bootstrap.cc:654] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:18.056268 13045 tablet_bootstrap.cc:492] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: No bootstrap required, opened a new log
I20260620 11:38:18.056331 13043 tablet_bootstrap.cc:492] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73: No bootstrap required, opened a new log
I20260620 11:38:18.056336 13045 ts_tablet_manager.cc:1397] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:18.056416 13043 ts_tablet_manager.cc:1397] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:38:18.056485 13045 raft_consensus.cc:348] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.056566 13045 raft_consensus.cc:374] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:18.056581 13045 raft_consensus.cc:729] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089d79306e8d4dfabd921e78de330db9, State: Initialized, Role: FOLLOWER
I20260620 11:38:18.056638 13045 consensus_queue.cc:260] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.056607 13043 raft_consensus.cc:348] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.056684 13043 raft_consensus.cc:374] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:18.056710 13043 raft_consensus.cc:729] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d80693a30fb469cb49af5d9122d0c73, State: Initialized, Role: FOLLOWER
I20260620 11:38:18.056785 13043 consensus_queue.cc:260] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.056815 13045 ts_tablet_manager.cc:1428] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:18.057081 13043 ts_tablet_manager.cc:1428] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:18.057127 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.056991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:54035)
0620 11:38:18.057055 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.057093 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 11:38:18.057418 13044 tablet_bootstrap.cc:492] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: No bootstrap required, opened a new log
I20260620 11:38:18.057466 13044 ts_tablet_manager.cc:1397] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:38:18.057523 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.057424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:52483)
0620 11:38:18.057466 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.057490 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 11:38:18.057567 12802 heartbeater.cc:503] Master 127.5.122.189:33391 was elected leader, sending a full tablet report...
I20260620 11:38:18.057611 13044 raft_consensus.cc:348] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.057693 13044 raft_consensus.cc:374] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:18.057718 13044 raft_consensus.cc:729] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6bc15329f242cf8a865bb76548b697, State: Initialized, Role: FOLLOWER
I20260620 11:38:18.057758 13044 consensus_queue.cc:260] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.057823 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.057747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:36299)
0620 11:38:18.057782 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.057802 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 11:38:18.057931 13044 ts_tablet_manager.cc:1428] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:18.057987 12665 heartbeater.cc:503] Master 127.5.122.189:33391 was elected leader, sending a full tablet report...
I20260620 11:38:18.058256 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.058160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:59463)
0620 11:38:18.058201 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.058227 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 11:38:18.058938 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.058766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:45235)
0620 11:38:18.058876 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.058907 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260620 11:38:18.059229 12938 heartbeater.cc:503] Master 127.5.122.189:33391 was elected leader, sending a full tablet report...
I20260620 11:38:18.059284 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.059175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:50785)
0620 11:38:18.059209 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.059235 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260620 11:38:18.308158 13051 raft_consensus.cc:482] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:18.308269 13051 raft_consensus.cc:504] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.308655 13051 leader_election.cc:288] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 089d79306e8d4dfabd921e78de330db9 (127.5.122.130:34363), 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:18.311966 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.308836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:36139 (local address 127.5.122.131:34983)
0620 11:38:18.309111 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.309122 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:18.309183 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:18.309375 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:18.309377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:18.309379 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:18.309445 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.309447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.310199 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.310200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.311071 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.311073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.311699 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.311700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.311777 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.311780 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:18.311837 (+ 57us) client_negotiation.cc:240] Negotiation successful
0620 11:38:18.311877 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":61,"thread_start_us":130,"threads_started":1}
I20260620 11:38:18.311959 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.308929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:34983 (local address 127.5.122.129:36139)
0620 11:38:18.308976 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:18.308978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:18.309197 (+ 219us) server_negotiation.cc:407] Connection header received
0620 11:38:18.309236 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:18.309237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:18.309248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:18.309314 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:18.309529 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.310077 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.311193 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.311618 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.311679 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.311687 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:18.311846 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 11:38:18.311891 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 11:38:18.312070 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.308905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:41019 (local address 127.5.122.130:34363)
0620 11:38:18.308984 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:18.308986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:18.308997 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:38:18.309059 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:18.309060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:18.309071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:18.309131 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:18.309376 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.310019 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.311162 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.311699 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.311775 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.311786 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:18.311965 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 11:38:18.312010 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 11:38:18.312070 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.308836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:34363 (local address 127.5.122.131:41019)
0620 11:38:18.308911 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.308924 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:18.308988 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:18.309148 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:18.309149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:18.309152 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:18.309264 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.309267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.310102 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.310103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.311069 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:18.311072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.311804 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.311806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:18.311884 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.311888 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:18.311955 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 11:38:18.311995 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":2}
I20260620 11:38:18.312482 12754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d79507399cc4569bc42f63dc4c1b010" candidate_uuid: "9d6bc15329f242cf8a865bb76548b697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089d79306e8d4dfabd921e78de330db9" is_pre_election: true
I20260620 11:38:18.312476 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d79507399cc4569bc42f63dc4c1b010" candidate_uuid: "9d6bc15329f242cf8a865bb76548b697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d80693a30fb469cb49af5d9122d0c73" is_pre_election: true
I20260620 11:38:18.312601 12617 raft_consensus.cc:2454] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d6bc15329f242cf8a865bb76548b697 in term 0.
I20260620 11:38:18.312618 12754 raft_consensus.cc:2454] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d6bc15329f242cf8a865bb76548b697 in term 0.
I20260620 11:38:18.312808 12824 leader_election.cc:302] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d80693a30fb469cb49af5d9122d0c73, 9d6bc15329f242cf8a865bb76548b697; no voters:
I20260620 11:38:18.313009 13051 raft_consensus.cc:2781] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:18.313047 13051 raft_consensus.cc:482] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:18.313061 13051 raft_consensus.cc:3037] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:18.313760 13051 raft_consensus.cc:504] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.313906 13051 leader_election.cc:288] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [CANDIDATE]: Term 1 election: Requested vote from peers 089d79306e8d4dfabd921e78de330db9 (127.5.122.130:34363), 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139)
I20260620 11:38:18.314090 12754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d79507399cc4569bc42f63dc4c1b010" candidate_uuid: "9d6bc15329f242cf8a865bb76548b697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "089d79306e8d4dfabd921e78de330db9"
I20260620 11:38:18.314190 12754 raft_consensus.cc:3037] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:18.314141 12617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d79507399cc4569bc42f63dc4c1b010" candidate_uuid: "9d6bc15329f242cf8a865bb76548b697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d80693a30fb469cb49af5d9122d0c73"
I20260620 11:38:18.314234 12617 raft_consensus.cc:3037] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:18.314854 12617 raft_consensus.cc:2454] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d6bc15329f242cf8a865bb76548b697 in term 1.
I20260620 11:38:18.314932 12754 raft_consensus.cc:2454] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d6bc15329f242cf8a865bb76548b697 in term 1.
I20260620 11:38:18.315115 12824 leader_election.cc:302] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d80693a30fb469cb49af5d9122d0c73, 9d6bc15329f242cf8a865bb76548b697; no voters:
I20260620 11:38:18.315244 13051 raft_consensus.cc:2781] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:18.315402 13051 raft_consensus.cc:686] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 1 LEADER]: Becoming Leader. State: Replica: 9d6bc15329f242cf8a865bb76548b697, State: Running, Role: LEADER
I20260620 11:38:18.315485 13051 consensus_queue.cc:237] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } }
I20260620 11:38:18.315729 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.315625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:46511)
0620 11:38:18.315671 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.315699 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 11:38:18.316223 12297 catalog_manager.cc:5495] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "9d6bc15329f242cf8a865bb76548b697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d6bc15329f242cf8a865bb76548b697" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 40381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 } health_report { overall_health: UNKNOWN } } }
I20260620 11:38:18.338482 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.335812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33078 (local address 127.5.122.131:40381)
0620 11:38:18.335872 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:18.335874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:18.335882 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:18.335925 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:18.335926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:18.335954 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:18.336062 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:18.336183 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.336730 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.337638 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:18.337836 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:18.337913 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:18.338114 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:18.338239 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:18.338320 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:18.338369 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 11:38:18.338393 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 11:38:18.339792 12617 raft_consensus.cc:1261] T 2d79507399cc4569bc42f63dc4c1b010 P 2d80693a30fb469cb49af5d9122d0c73 [term 1 FOLLOWER]: Refusing update from remote peer 9d6bc15329f242cf8a865bb76548b697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:18.339931 12754 raft_consensus.cc:1261] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 1 FOLLOWER]: Refusing update from remote peer 9d6bc15329f242cf8a865bb76548b697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:18.340111 13053 consensus_queue.cc:1035] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d80693a30fb469cb49af5d9122d0c73" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:18.340267 13051 consensus_queue.cc:1035] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "089d79306e8d4dfabd921e78de330db9" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:18.341022 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.340901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:58375)
0620 11:38:18.340948 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.340982 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260620 11:38:18.341610 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.341498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:56771)
0620 11:38:18.341542 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.341575 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 11:38:18.342658 13022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.342547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.131:56961)
0620 11:38:18.342588 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.342623 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260620 11:38:18.343128 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.343022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.129:35113)
0620 11:38:18.343066 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.343098 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 11:38:18.343637 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.343538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38037 (local address 127.5.122.130:46421)
0620 11:38:18.343588 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.343616 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:38037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 11:38:18.345906 12296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:18.346028 12296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:18.347384 12296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 207ee50512ab407abb46e19afe1e2291: Sending DeleteTablet for 3 replicas of tablet 2d79507399cc4569bc42f63dc4c1b010
I20260620 11:38:18.347735 12597 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d79507399cc4569bc42f63dc4c1b010 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:18 UTC) from {username='oryx', principal='oryx/127.5.122.189@KRBTEST.COM'} at 127.0.0.1:51098
I20260620 11:38:18.347899 12870 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d79507399cc4569bc42f63dc4c1b010 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:18 UTC) from {username='oryx', principal='oryx/127.5.122.189@KRBTEST.COM'} at 127.0.0.1:33062
I20260620 11:38:18.348026 12734 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d79507399cc4569bc42f63dc4c1b010 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:18 UTC) from {username='oryx', principal='oryx/127.5.122.189@KRBTEST.COM'} at 127.0.0.1:41932
I20260620 11:38:18.348109 13067 tablet_replica.cc:330] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: stopping tablet replica
I20260620 11:38:18.348172 13067 raft_consensus.cc:2229] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:38:18.348269 13067 raft_consensus.cc:2258] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:18.348272 13068 tablet_replica.cc:330] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: stopping tablet replica
I20260620 11:38:18.348417 13068 raft_consensus.cc:2229] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:18.348503 13068 raft_consensus.cc:2258] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:18.348588 13067 ts_tablet_manager.cc:1905] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:18.348589 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 12445
I20260620 11:38:18.348922 13068 ts_tablet_manager.cc:1905] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:18.350972 13067 ts_tablet_manager.cc:1918] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:18.351040 13067 log.cc:1196] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/2d79507399cc4569bc42f63dc4c1b010
I20260620 11:38:18.351187 13068 ts_tablet_manager.cc:1918] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:18.351274 13067 ts_tablet_manager.cc:1939] T 2d79507399cc4569bc42f63dc4c1b010 P 9d6bc15329f242cf8a865bb76548b697: Deleting consensus metadata
I20260620 11:38:18.351300 13068 log.cc:1196] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/2d79507399cc4569bc42f63dc4c1b010
I20260620 11:38:18.351595 12285 catalog_manager.cc:4841] TS 9d6bc15329f242cf8a865bb76548b697 (127.5.122.131:40381): tablet 2d79507399cc4569bc42f63dc4c1b010 (table test-table [id=cb53f63ec63544ed93c84955902a8f55]) successfully deleted
I20260620 11:38:18.351727 13068 ts_tablet_manager.cc:1939] T 2d79507399cc4569bc42f63dc4c1b010 P 089d79306e8d4dfabd921e78de330db9: Deleting consensus metadata
I20260620 11:38:18.352101 12282 catalog_manager.cc:4841] TS 089d79306e8d4dfabd921e78de330db9 (127.5.122.130:34363): tablet 2d79507399cc4569bc42f63dc4c1b010 (table test-table [id=cb53f63ec63544ed93c84955902a8f55]) successfully deleted
I20260620 11:38:18.355687 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:18.355558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:36139 (local address 127.0.0.1:51114)
0620 11:38:18.355606 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:18.355639 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:36139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260620 11:38:18.355854 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 12671
W20260620 11:38:18.355907 12283 catalog_manager.cc:4584] TS 2d80693a30fb469cb49af5d9122d0c73 (127.5.122.129:36139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d79507399cc4569bc42f63dc4c1b010: Network error: Client connection negotiation failed: client connection to 127.5.122.129:36139: connect: Connection refused (error 111)
I20260620 11:38:18.362681 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 12807
I20260620 11:38:18.368242 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 12164
I20260620 11:38:18.373219 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 12352
2026-06-20T11:38:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6826 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:18 dist-test-slave-szdr krb5kdc[13071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:18 dist-test-slave-szdr krb5kdc[13071](info): set up 2 sockets
Jun 20 11:38:18 dist-test-slave-szdr krb5kdc[13071](info): commencing operation
krb5kdc: starting...
W20260620 11:38:19.422609 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:19 dist-test-slave-szdr krb5kdc[13071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab.
Entry for principal oryx/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab.
I20260620 11:38:19.549024 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:44709
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab
--principal=oryx/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:44709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:19.634531 13084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:19.634807 13084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:19.634850 13084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:19.636137 13084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:19.636215 13084 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 11:38:19.636240 13084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:19.636262 13084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:19.636296 13084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:19.636324 13084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:19.637988 13084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:44709
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.190.keytab
--principal=oryx/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:19.638289 13084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:19.638546 13084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:19.643632 13090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:19.643641 13092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:19.643668 13089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:19.643909 13084 server_base.cc:1034] running on GCE node
I20260620 11:38:19.644196 13084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:19.644515 13084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:19.645768 13084 hybrid_clock.cc:648] HybridClock initialized: now 1781955499645746 us; error 49 us; skew 500 ppm
Jun 20 11:38:19 dist-test-slave-szdr krb5kdc[13071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955499, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:19.648620 13084 init.cc:377] Logged in from keytab as oryx/127.5.122.190@KRBTEST.COM (short username oryx)
I20260620 11:38:19.649096 13084 webserver.cc:460] Webserver started at http://127.5.122.190:36947/ using document root <none> and password file <none>
I20260620 11:38:19.649356 13084 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:19.649428 13084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:19.649578 13084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:19.650746 13084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "dafcaffd92c74db3bd67ce19548854ae"
format_stamp: "Formatted at 2026-06-20 11:38:19 on dist-test-slave-szdr"
server_key: "a7bc26644f14f9d4470c471d7dc322aa"
server_key_iv: "f214f516040a0bfb6a9c69a6dbc04992"
server_key_version: "encryptionkey@0"
I20260620 11:38:19.651132 13084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "dafcaffd92c74db3bd67ce19548854ae"
format_stamp: "Formatted at 2026-06-20 11:38:19 on dist-test-slave-szdr"
server_key: "a7bc26644f14f9d4470c471d7dc322aa"
server_key_iv: "f214f516040a0bfb6a9c69a6dbc04992"
server_key_version: "encryptionkey@0"
I20260620 11:38:19.652594 13084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:19.653532 13099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:19.653734 13084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:19.653863 13084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "dafcaffd92c74db3bd67ce19548854ae"
format_stamp: "Formatted at 2026-06-20 11:38:19 on dist-test-slave-szdr"
server_key: "a7bc26644f14f9d4470c471d7dc322aa"
server_key_iv: "f214f516040a0bfb6a9c69a6dbc04992"
server_key_version: "encryptionkey@0"
I20260620 11:38:19.653947 13084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:19.675252 13084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:19.677794 13084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:19.677913 13084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:19.682982 13084 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:44709
I20260620 11:38:19.683071 13161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:44709 every 8 connection(s)
I20260620 11:38:19.683442 13084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:19.684190 13162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:19.684655 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 13084
I20260620 11:38:19.684736 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:19.685007 5610 external_mini_cluster.cc:1428] Setting key 8d960c4e653ed3fe6d266d3757e90880
I20260620 11:38:19.687458 13162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Bootstrap starting.
I20260620 11:38:19.688185 13162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:19.688498 13162 log.cc:824] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:19.689251 13162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: No bootstrap required, opened a new log
I20260620 11:38:19.691496 13162 raft_consensus.cc:348] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafcaffd92c74db3bd67ce19548854ae" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44709 } }
I20260620 11:38:19.691645 13162 raft_consensus.cc:374] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:19.691668 13162 raft_consensus.cc:729] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafcaffd92c74db3bd67ce19548854ae, State: Initialized, Role: FOLLOWER
I20260620 11:38:19.691846 13162 consensus_queue.cc:260] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafcaffd92c74db3bd67ce19548854ae" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44709 } }
I20260620 11:38:19.691985 13162 raft_consensus.cc:388] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:19.692037 13162 raft_consensus.cc:482] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:19.692092 13162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 0 FOLLOWER]: Advancing to term 1
Jun 20 11:38:19 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:19.692818 13162 raft_consensus.cc:504] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafcaffd92c74db3bd67ce19548854ae" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44709 } }
I20260620 11:38:19.692942 13162 leader_election.cc:302] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dafcaffd92c74db3bd67ce19548854ae; no voters:
I20260620 11:38:19.693141 13162 leader_election.cc:288] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:19.693192 13167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:19.693361 13167 raft_consensus.cc:686] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [term 1 LEADER]: Becoming Leader. State: Replica: dafcaffd92c74db3bd67ce19548854ae, State: Running, Role: LEADER
I20260620 11:38:19.693480 13167 consensus_queue.cc:237] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafcaffd92c74db3bd67ce19548854ae" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44709 } }
I20260620 11:38:19.693482 13162 sys_catalog.cc:564] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:19.694300 13169 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader dafcaffd92c74db3bd67ce19548854ae. Latest consensus state: current_term: 1 leader_uuid: "dafcaffd92c74db3bd67ce19548854ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafcaffd92c74db3bd67ce19548854ae" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44709 } } }
I20260620 11:38:19.694491 13169 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:19.694722 13168 sys_catalog.cc:455] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dafcaffd92c74db3bd67ce19548854ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dafcaffd92c74db3bd67ce19548854ae" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 44709 } } }
I20260620 11:38:19.694878 13168 sys_catalog.cc:458] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:19.694773 13173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:19.695565 13173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:19.697465 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:19.685618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.5.122.190:44709)
0620 11:38:19.685819 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:19.685830 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:19.685867 (+ 37us) server_negotiation.cc:407] Connection header received
0620 11:38:19.686441 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:19.686450 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:19.686513 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:19.686678 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:19.687285 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.688686 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.689560 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.690232 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.690385 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:19.692846 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:19.692850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:19.692850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:19.692863 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:19.695539 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:19.695993 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:19.695996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:19.695997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:19.696104 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:19.696369 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:19.696370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:19.696371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:19.696678 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:19.696829 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:19.697105 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 11:38:19.697265 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260620 11:38:19.697482 13173 catalog_manager.cc:1347] Generated new cluster ID: 645948aa8ce04ed88dc9d7b05182ada3
I20260620 11:38:19.697530 13173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:19.712994 13173 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:19.713594 13173 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:19.715824 13173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Generated new TSK 0
I20260620 11:38:19.716066 13173 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab.
Entry for principal oryx/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab.
I20260620 11:38:19.748459 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:44709
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab
--principal=oryx/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:19.836308 13188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:19.836545 13188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:19.836573 13188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:19.837914 13188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:19.838029 13188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:19.838066 13188 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:38:19.838120 13188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:19.839782 13188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab
--principal=oryx/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:19.840094 13188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:19.840346 13188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:38:19.846592 13188 server_base.cc:1034] running on GCE node
W20260620 11:38:19.846742 13196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:19.846761 13193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:19.846956 13194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:19.847187 13188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:19.847505 13188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:19.848682 13188 hybrid_clock.cc:648] HybridClock initialized: now 1781955499848663 us; error 34 us; skew 500 ppm
Jun 20 11:38:19 dist-test-slave-szdr krb5kdc[13071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955499, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:19.851532 13188 init.cc:377] Logged in from keytab as oryx/127.5.122.129@KRBTEST.COM (short username oryx)
I20260620 11:38:19.852036 13188 webserver.cc:460] Webserver started at http://127.5.122.129:33985/ using document root <none> and password file <none>
I20260620 11:38:19.852246 13188 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:19.852288 13188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:19.852382 13188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:19.853422 13188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "e69dbe66e24246e1bd20f5342952276c"
format_stamp: "Formatted at 2026-06-20 11:38:19 on dist-test-slave-szdr"
server_key: "7c27e18e4a0d281a4206d6a32d2e26d1"
server_key_iv: "93d28ee2b77149718f866e27d3738d86"
server_key_version: "encryptionkey@0"
I20260620 11:38:19.853718 13188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "e69dbe66e24246e1bd20f5342952276c"
format_stamp: "Formatted at 2026-06-20 11:38:19 on dist-test-slave-szdr"
server_key: "7c27e18e4a0d281a4206d6a32d2e26d1"
server_key_iv: "93d28ee2b77149718f866e27d3738d86"
server_key_version: "encryptionkey@0"
I20260620 11:38:19.855230 13188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:19.855959 13203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:19.856161 13188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:19.856273 13188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "e69dbe66e24246e1bd20f5342952276c"
format_stamp: "Formatted at 2026-06-20 11:38:19 on dist-test-slave-szdr"
server_key: "7c27e18e4a0d281a4206d6a32d2e26d1"
server_key_iv: "93d28ee2b77149718f866e27d3738d86"
server_key_version: "encryptionkey@0"
I20260620 11:38:19.856364 13188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:19.871698 13188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:19.874483 13188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:19.874723 13188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:19.875388 13188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:19.875458 13188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:19.875494 13188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:19.875523 13188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 11:38:19 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955499, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:19.883352 13188 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:45841
I20260620 11:38:19.883399 13318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:45841 every 8 connection(s)
I20260620 11:38:19.883795 13188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:19.884399 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 13188
I20260620 11:38:19.884539 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:19.884780 5610 external_mini_cluster.cc:1428] Setting key 560dcba460270230682cfc8907040cfb
I20260620 11:38:19.886322 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:19.875953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:35685 (local address 127.5.122.190:44709)
0620 11:38:19.876030 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:19.876033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:19.876309 (+ 276us) server_negotiation.cc:407] Connection header received
0620 11:38:19.877096 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:19.877098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:19.877109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:19.877195 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:19.878352 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.878997 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.880100 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.880333 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.880419 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:19.882930 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:19.882932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:19.882932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:19.882942 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:19.884712 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:19.885220 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:19.885221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:19.885222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:19.885287 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:19.885633 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:19.885633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:19.885634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:19.885747 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:19.885800 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:19.886182 (+ 382us) server_negotiation.cc:299] Negotiation successful
0620 11:38:19.886258 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 11:38:19.886668 13212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:19.875936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44709 (local address 127.5.122.129:35685)
0620 11:38:19.876147 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:19.876202 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:19.876960 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:19.877308 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:19.877314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:19.877825 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:19.878206 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:19.878215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.879115 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.879116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:19.879997 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:19.880000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.880445 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.880447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:19.880591 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:19.881179 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:19.881189 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:19.882817 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:19.884795 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:19.884803 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:19.884805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:19.885135 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:19.885358 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:19.885374 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:19.885374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:19.885563 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:19.885816 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:19.885824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:19.886027 (+ 203us) client_negotiation.cc:769] Sending connection context
0620 11:38:19.886157 (+ 130us) client_negotiation.cc:240] Negotiation successful
0620 11:38:19.886358 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
I20260620 11:38:19.887256 13319 heartbeater.cc:347] Connected to a master server at 127.5.122.190:44709
I20260620 11:38:19.887367 13319 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:19.887563 13319 heartbeater.cc:511] Master 127.5.122.190:44709 requested a full tablet report, sending...
I20260620 11:38:19.888168 13115 ts_manager.cc:194] Registered new tserver with Master: e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841)
I20260620 11:38:19.888816 13115 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:35685
WARNING: no policy specified for oryx/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.130@KRBTEST.COM" created.
I20260620 11:38:19.896641 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:19.890220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44709 (local address 127.5.122.129:40559)
0620 11:38:19.890426 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:19.890436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:19.890517 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:19.890639 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:19.890639 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:19.890840 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:19.890900 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:19.890901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.891572 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.891573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:19.892098 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:19.892099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.892469 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.892470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:19.892532 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:19.893046 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:19.893056 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:19.893492 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:19.895431 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:19.895432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:19.895432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:19.895613 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:19.895847 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:19.895848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:19.895848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:19.895925 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:19.896275 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:19.896276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:19.896334 (+ 58us) client_negotiation.cc:769] Sending connection context
0620 11:38:19.896414 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 11:38:19.896538 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":110,"threads_started":1}
I20260620 11:38:19.896637 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:19.890293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:40559 (local address 127.5.122.190:44709)
0620 11:38:19.890361 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:19.890363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:19.890501 (+ 138us) server_negotiation.cc:407] Connection header received
0620 11:38:19.890567 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:19.890568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:19.890578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:19.890635 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:19.890977 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.891464 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.892187 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:19.892393 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:19.892479 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:19.893592 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:19.893593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:19.893594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:19.893601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:19.895353 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:19.895723 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:19.895724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:19.895724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:19.895787 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:19.896075 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:19.896076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:19.896076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:19.896193 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:19.896281 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:19.896446 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 11:38:19.896534 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab.
Entry for principal oryx/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab.
I20260620 11:38:19.913431 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:44709
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab
--principal=oryx/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:19.998588 13328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:19.998821 13328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:19.998848 13328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:20.000036 13328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:20.000082 13328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:20.000104 13328 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:38:20.000123 13328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:20.001752 13328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab
--principal=oryx/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:20.002066 13328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:20.002288 13328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:20.007550 13336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:20.007558 13334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:20.007615 13328 server_base.cc:1034] running on GCE node
W20260620 11:38:20.007576 13333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:20.008010 13328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:20.008270 13328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:20.009464 13328 hybrid_clock.cc:648] HybridClock initialized: now 1781955500009455 us; error 39 us; skew 500 ppm
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:20.012320 13328 init.cc:377] Logged in from keytab as oryx/127.5.122.130@KRBTEST.COM (short username oryx)
I20260620 11:38:20.012749 13328 webserver.cc:460] Webserver started at http://127.5.122.130:33319/ using document root <none> and password file <none>
I20260620 11:38:20.013000 13328 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:20.013047 13328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:20.013185 13328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:20.014376 13328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "327681b01a0846c8ab76dbedb1916c64"
format_stamp: "Formatted at 2026-06-20 11:38:20 on dist-test-slave-szdr"
server_key: "bbc8b6fa4cfa71f3b2e982676716d22a"
server_key_iv: "9f24babee7607f973ca3616408af538d"
server_key_version: "encryptionkey@0"
I20260620 11:38:20.014765 13328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "327681b01a0846c8ab76dbedb1916c64"
format_stamp: "Formatted at 2026-06-20 11:38:20 on dist-test-slave-szdr"
server_key: "bbc8b6fa4cfa71f3b2e982676716d22a"
server_key_iv: "9f24babee7607f973ca3616408af538d"
server_key_version: "encryptionkey@0"
I20260620 11:38:20.016184 13328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:20.017042 13343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.017263 13328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:20.017383 13328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "327681b01a0846c8ab76dbedb1916c64"
format_stamp: "Formatted at 2026-06-20 11:38:20 on dist-test-slave-szdr"
server_key: "bbc8b6fa4cfa71f3b2e982676716d22a"
server_key_iv: "9f24babee7607f973ca3616408af538d"
server_key_version: "encryptionkey@0"
I20260620 11:38:20.017490 13328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:20.029810 13328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:20.032147 13328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:20.032321 13328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:20.032934 13328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:20.032979 13328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.033003 13328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:20.033016 13328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.040107 13328 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:38627
I20260620 11:38:20.040210 13458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:38627 every 8 connection(s)
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:20.040542 13328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:20.043735 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.033501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:56441 (local address 127.5.122.190:44709)
0620 11:38:20.033577 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.033580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.033752 (+ 172us) server_negotiation.cc:407] Connection header received
0620 11:38:20.034364 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.034366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.034379 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.034420 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.035195 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.036075 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.037691 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.038075 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.038171 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.041003 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:20.041005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:20.041005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:20.041013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:20.042535 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.042913 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.042913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.042914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.042974 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.043180 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.043181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.043181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.043287 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:20.043337 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.043594 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.043669 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:38:20.043911 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.033449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44709 (local address 127.5.122.130:56441)
0620 11:38:20.033611 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.033655 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.034246 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.034524 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.034528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.034837 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:20.035088 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.035095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.036161 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.036163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.037612 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.037616 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.038181 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.038183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.038345 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.038936 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:20.038947 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:20.040873 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:20.042610 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.042616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.042618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.042834 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.043031 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.043031 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.043032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.043128 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.043340 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:20.043344 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:20.043479 (+ 135us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.043566 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.043692 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260620 11:38:20.044422 13459 heartbeater.cc:347] Connected to a master server at 127.5.122.190:44709
I20260620 11:38:20.044528 13459 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:20.044716 13459 heartbeater.cc:511] Master 127.5.122.190:44709 requested a full tablet report, sending...
I20260620 11:38:20.045145 13116 ts_manager.cc:194] Registered new tserver with Master: 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627)
I20260620 11:38:20.045562 13116 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:56441
I20260620 11:38:20.048641 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 13328
I20260620 11:38:20.048779 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:20.049047 5610 external_mini_cluster.cc:1428] Setting key 91e29cd066d05bd998c3a84d4d3cf800
I20260620 11:38:20.052138 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.046241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44709 (local address 127.5.122.130:36843)
0620 11:38:20.046368 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.046379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.046440 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.046596 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.046597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.046773 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:20.046824 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.046825 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.047472 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.047473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.047994 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.047996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.048362 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.048363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.048427 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.048906 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:20.048911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:20.049334 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:20.051034 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.051035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.051036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.051279 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.051488 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.051488 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.051489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.051560 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.051894 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:20.051895 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:20.051933 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.051985 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.052069 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260620 11:38:20.052237 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.046309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:36843 (local address 127.5.122.190:44709)
0620 11:38:20.046354 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.046356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.046436 (+ 80us) server_negotiation.cc:407] Connection header received
0620 11:38:20.046516 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.046517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.046526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.046579 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.046897 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.047375 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.048078 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.048287 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.048363 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.049430 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:20.049431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:20.049431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:20.049438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:20.050922 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.051355 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.051356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.051357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.051422 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.051621 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.051622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.051623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.051830 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:20.051899 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.052017 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.052098 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
WARNING: no policy specified for oryx/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab.
Entry for principal oryx/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab.
I20260620 11:38:20.076514 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:44709
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab
--principal=oryx/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:20.162505 13468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:20.162757 13468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:20.162786 13468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:20.163998 13468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:20.164043 13468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:20.164064 13468 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 11:38:20.164083 13468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:20.165657 13468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab
--principal=oryx/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.122.190:44709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:20.165885 13468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:20.166172 13468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:20.171792 13476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:20.171794 13474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:20.171792 13473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:20.172034 13468 server_base.cc:1034] running on GCE node
I20260620 11:38:20.172333 13468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:20.172598 13468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:20.173761 13468 hybrid_clock.cc:648] HybridClock initialized: now 1781955500173744 us; error 26 us; skew 500 ppm
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:20.176537 13468 init.cc:377] Logged in from keytab as oryx/127.5.122.131@KRBTEST.COM (short username oryx)
I20260620 11:38:20.176959 13468 webserver.cc:460] Webserver started at http://127.5.122.131:38319/ using document root <none> and password file <none>
I20260620 11:38:20.177176 13468 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:20.177220 13468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:20.177317 13468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:20.178444 13468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "2fac61481ef94ba7a1d9860c46c31907"
format_stamp: "Formatted at 2026-06-20 11:38:20 on dist-test-slave-szdr"
server_key: "b56a4394c4175ca02e8bc0fadbd3d586"
server_key_iv: "a9101b68758adbac66a76e1ce48e6e24"
server_key_version: "encryptionkey@0"
I20260620 11:38:20.178762 13468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "2fac61481ef94ba7a1d9860c46c31907"
format_stamp: "Formatted at 2026-06-20 11:38:20 on dist-test-slave-szdr"
server_key: "b56a4394c4175ca02e8bc0fadbd3d586"
server_key_iv: "a9101b68758adbac66a76e1ce48e6e24"
server_key_version: "encryptionkey@0"
I20260620 11:38:20.180183 13468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:20.180886 13483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.181058 13468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:20.181181 13468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "2fac61481ef94ba7a1d9860c46c31907"
format_stamp: "Formatted at 2026-06-20 11:38:20 on dist-test-slave-szdr"
server_key: "b56a4394c4175ca02e8bc0fadbd3d586"
server_key_iv: "a9101b68758adbac66a76e1ce48e6e24"
server_key_version: "encryptionkey@0"
I20260620 11:38:20.181267 13468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:20.199759 13468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:20.201930 13468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:20.202191 13468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:20.202755 13468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:20.202834 13468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.202873 13468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:20.202901 13468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.210304 13468 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:46665
I20260620 11:38:20.210433 13598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:46665 every 8 connection(s)
I20260620 11:38:20.210739 13468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:20.212128 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 13468
I20260620 11:38:20.212263 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:20.212499 5610 external_mini_cluster.cc:1428] Setting key 9f4069beee3d768a04a1ead0f1f9ffac
I20260620 11:38:20.215294 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.203248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:48389 (local address 127.5.122.190:44709)
0620 11:38:20.203292 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.203294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.203542 (+ 248us) server_negotiation.cc:407] Connection header received
0620 11:38:20.204097 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.204099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.204112 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.204158 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.205179 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.206080 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.207268 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.207654 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.207753 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.212219 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:20.212221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:20.212221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:20.212232 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:20.214040 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.214486 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.214487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.214488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.214545 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.214766 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.214767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.214767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.214880 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:20.214931 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.215163 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.215228 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:38:20.215531 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.203258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44709 (local address 127.5.122.131:48389)
0620 11:38:20.203407 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.203445 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.203982 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.204255 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.204260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.204718 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:20.205046 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.205054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.206187 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.206189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.207153 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.207155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.207783 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.207785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.207927 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.208910 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:20.208919 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:20.212073 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:20.214154 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.214161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.214163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.214418 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.214604 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.214604 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.214604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.214712 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.214921 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:20.214924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:20.215050 (+ 126us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.215147 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.215288 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":82,"threads_started":1}
I20260620 11:38:20.216148 13599 heartbeater.cc:347] Connected to a master server at 127.5.122.190:44709
I20260620 11:38:20.216241 13599 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:20.216598 13599 heartbeater.cc:511] Master 127.5.122.190:44709 requested a full tablet report, sending...
I20260620 11:38:20.217063 13115 ts_manager.cc:194] Registered new tserver with Master: 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665)
I20260620 11:38:20.217635 13115 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:48389
I20260620 11:38:20.222646 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.217288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44709 (local address 127.5.122.131:38947)
0620 11:38:20.217432 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.217458 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.217518 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.217628 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.217629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.217847 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:20.217899 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.217901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.218611 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.218611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.219094 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.219095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.219412 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.219413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.219459 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.219869 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:20.219873 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:20.220169 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:20.221645 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.221646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.221647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.221861 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.222133 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.222134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.222134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.222190 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.222391 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:20.222391 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:20.222435 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.222481 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.222566 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260620 11:38:20.222631 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.217377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:38947 (local address 127.5.122.190:44709)
0620 11:38:20.217419 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.217421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.217508 (+ 87us) server_negotiation.cc:407] Connection header received
0620 11:38:20.217571 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.217571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.217579 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.217623 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.218043 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.218514 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.219176 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.219351 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.219404 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.220246 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:20.220247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:20.220247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:20.220253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:20.221573 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.221959 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.221960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.222000 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.222063 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.222239 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.222240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.222240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.222346 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:20.222398 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.222498 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.222559 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:38:20.224392 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 11:38:20.246960 13165 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:20.240858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.5.122.190:44709)
0620 11:38:20.240925 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.240926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.240941 (+ 15us) server_negotiation.cc:407] Connection header received
0620 11:38:20.241008 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.241009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.241019 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.241073 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.241622 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.242240 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.242999 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.243177 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.243233 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.246850 (+ 3617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44824: BlockingRecv error: recv got EOF from 127.0.0.1:44824 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:20.255451 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.247802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.5.122.190:44709)
0620 11:38:20.247968 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.247969 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.247977 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:20.248009 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.248010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.248018 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.248069 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.248563 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.249124 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.250143 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.250369 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.250439 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.252841 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:20.252842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:20.252842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:20.252850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:20.254474 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.254819 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.254820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.254820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.254876 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.255043 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.255044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.255044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.255154 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:20.255231 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.255320 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.255380 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260620 11:38:20.257356 13115 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:20.259284 13115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:20.266680 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.262351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42754 (local address 127.5.122.131:46665)
0620 11:38:20.262620 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.262627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.262641 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:38:20.262723 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.262725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.262740 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.262819 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.263151 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.263997 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.265357 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.266183 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.266289 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.266372 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.266521 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.266565 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
I20260620 11:38:20.266788 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.262232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:38627 (local address 127.0.0.1:33044)
0620 11:38:20.262423 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.262456 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.262525 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.263138 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.263150 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.263152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.263222 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.263224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.264600 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.264601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.265572 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.265573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.266524 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.266525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.266587 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.266591 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.266623 (+ 32us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.266665 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":5,"spinlock_wait_cycles":768,"thread_start_us":91,"threads_started":1}
I20260620 11:38:20.266664 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.262232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:46665 (local address 127.0.0.1:42754)
0620 11:38:20.262399 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.262456 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.262525 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.262904 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.262907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.262909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.263062 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.263064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.264082 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.264083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.265262 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.265265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.266292 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.266293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.266376 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.266397 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.266503 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.266566 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260620 11:38:20.267270 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.262487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:45841 (local address 127.0.0.1:37802)
0620 11:38:20.263173 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.263184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.263228 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.263509 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.263510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.263512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.263558 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.263559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.264753 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.264754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.265892 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.265895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.267027 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.267028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.267103 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.267108 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.267159 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.267200 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":88,"threads_started":1}
I20260620 11:38:20.267258 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.262696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.5.122.130:38627)
0620 11:38:20.262946 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.262951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.262962 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:38:20.263003 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.263006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.263027 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.263301 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.263366 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.264497 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.265654 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.266419 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.266711 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.267059 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.267101 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.267154 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260620 11:38:20.267575 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.262990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37802 (local address 127.5.122.129:45841)
0620 11:38:20.263337 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.263342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.263353 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:38:20.263391 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.263394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.263410 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.263490 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.263652 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.264664 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.266021 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.266846 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.266917 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.266990 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.267432 (+ 442us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.267473 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":92,"threads_started":1}
I20260620 11:38:20.267614 13533 tablet_service.cc:1467] Processing CreateTablet for tablet 27d15f3d64e84dc997f5fc5d4ec260be (DEFAULT_TABLE table=test-table [id=f92ad90dd27a4ba88c7938331be5556d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.268039 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d15f3d64e84dc997f5fc5d4ec260be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.268188 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 27d15f3d64e84dc997f5fc5d4ec260be (DEFAULT_TABLE table=test-table [id=f92ad90dd27a4ba88c7938331be5556d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.268185 13253 tablet_service.cc:1467] Processing CreateTablet for tablet 27d15f3d64e84dc997f5fc5d4ec260be (DEFAULT_TABLE table=test-table [id=f92ad90dd27a4ba88c7938331be5556d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.268509 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d15f3d64e84dc997f5fc5d4ec260be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.268509 13253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27d15f3d64e84dc997f5fc5d4ec260be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.271005 13627 tablet_bootstrap.cc:492] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Bootstrap starting.
I20260620 11:38:20.271610 13627 tablet_bootstrap.cc:654] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.271863 13627 log.cc:824] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:20.272578 13627 tablet_bootstrap.cc:492] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: No bootstrap required, opened a new log
I20260620 11:38:20.272639 13627 ts_tablet_manager.cc:1397] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:38:20.272662 13629 tablet_bootstrap.cc:492] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Bootstrap starting.
I20260620 11:38:20.273027 13630 tablet_bootstrap.cc:492] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Bootstrap starting.
I20260620 11:38:20.273298 13629 tablet_bootstrap.cc:654] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.273628 13629 log.cc:824] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:20.273707 13630 tablet_bootstrap.cc:654] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.274111 13630 log.cc:824] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:20.274588 13629 tablet_bootstrap.cc:492] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: No bootstrap required, opened a new log
I20260620 11:38:20.274667 13629 ts_tablet_manager.cc:1397] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:38:20.274869 13630 tablet_bootstrap.cc:492] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: No bootstrap required, opened a new log
I20260620 11:38:20.274950 13630 ts_tablet_manager.cc:1397] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:38:20.275188 13627 raft_consensus.cc:348] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.275393 13627 raft_consensus.cc:374] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.275430 13627 raft_consensus.cc:729] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fac61481ef94ba7a1d9860c46c31907, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.275578 13627 consensus_queue.cc:260] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.275882 13627 ts_tablet_manager.cc:1428] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:38:20.276022 13599 heartbeater.cc:503] Master 127.5.122.190:44709 was elected leader, sending a full tablet report...
I20260620 11:38:20.276844 13630 raft_consensus.cc:348] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.277029 13630 raft_consensus.cc:374] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.277067 13630 raft_consensus.cc:729] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69dbe66e24246e1bd20f5342952276c, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.277201 13630 consensus_queue.cc:260] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.277175 13629 raft_consensus.cc:348] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.277346 13629 raft_consensus.cc:374] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.277392 13629 raft_consensus.cc:729] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.277499 13630 ts_tablet_manager.cc:1428] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 11:38:20.277650 13319 heartbeater.cc:503] Master 127.5.122.190:44709 was elected leader, sending a full tablet report...
I20260620 11:38:20.277525 13629 consensus_queue.cc:260] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.278141 13629 ts_tablet_manager.cc:1428] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:38:20.278241 13459 heartbeater.cc:503] Master 127.5.122.190:44709 was elected leader, sending a full tablet report...
W20260620 11:38:20.291256 13460 tablet.cc:2369] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:38:20.314977 13635 raft_consensus.cc:482] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:20.315277 13635 raft_consensus.cc:504] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.315852 13635 leader_election.cc:288] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665), e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841)
I20260620 11:38:20.319900 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.316071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:45841 (local address 127.5.122.130:44047)
0620 11:38:20.316130 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.316143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.316187 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.316341 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.316343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.316345 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.316423 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.316425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.317433 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.317434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.318784 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.318787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.319568 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.319569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.319686 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.319693 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.319757 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.319810 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":37}
I20260620 11:38:20.319929 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.316146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:44047 (local address 127.5.122.129:45841)
0620 11:38:20.316204 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.316206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.316216 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:20.316246 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.316247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.316256 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.316310 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.316514 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.317341 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.318904 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.319487 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.319553 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.319559 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.319811 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.319860 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 11:38:20.319954 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.316268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:45867 (local address 127.5.122.131:46665)
0620 11:38:20.316340 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.316342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.316351 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:20.316428 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.316432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.316440 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.316508 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.316658 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.317340 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.318903 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.319462 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.319549 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.319559 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.319811 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.319858 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 11:38:20.320067 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.316075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:46665 (local address 127.5.122.130:45867)
0620 11:38:20.316289 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.316299 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.316359 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.316514 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.316514 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.316516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.316583 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.316584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.317433 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.317434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.318784 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.318787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.319568 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.319569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.319686 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.319693 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.319757 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.319800 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":121,"thread_start_us":114,"threads_started":1}
I20260620 11:38:20.320338 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d15f3d64e84dc997f5fc5d4ec260be" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c" is_pre_election: true
I20260620 11:38:20.320607 13273 raft_consensus.cc:2454] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 0.
I20260620 11:38:20.320581 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d15f3d64e84dc997f5fc5d4ec260be" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907" is_pre_election: true
I20260620 11:38:20.320720 13553 raft_consensus.cc:2454] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 0.
I20260620 11:38:20.320875 13345 leader_election.cc:302] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 327681b01a0846c8ab76dbedb1916c64, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.321046 13635 raft_consensus.cc:2781] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:20.321116 13635 raft_consensus.cc:482] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:20.321133 13635 raft_consensus.cc:3037] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.322274 13635 raft_consensus.cc:504] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.322441 13635 leader_election.cc:288] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 election: Requested vote from peers 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665), e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841)
I20260620 11:38:20.322580 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d15f3d64e84dc997f5fc5d4ec260be" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907"
I20260620 11:38:20.322604 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27d15f3d64e84dc997f5fc5d4ec260be" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c"
I20260620 11:38:20.322664 13553 raft_consensus.cc:3037] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.322679 13273 raft_consensus.cc:3037] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.323310 13273 raft_consensus.cc:2454] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 1.
I20260620 11:38:20.323380 13553 raft_consensus.cc:2454] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 1.
I20260620 11:38:20.323503 13345 leader_election.cc:302] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 327681b01a0846c8ab76dbedb1916c64, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.323674 13635 raft_consensus.cc:2781] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:20.323844 13635 raft_consensus.cc:686] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 1 LEADER]: Becoming Leader. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Running, Role: LEADER
I20260620 11:38:20.323949 13635 consensus_queue.cc:237] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } }
I20260620 11:38:20.324923 13116 catalog_manager.cc:5495] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260620 11:38:20.339900 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.337414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.5.122.130:38627)
0620 11:38:20.337481 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.337483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.337491 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:20.337546 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.337547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.337566 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.337634 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:20.337816 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.338389 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.339192 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.339392 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.339464 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.339611 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.339714 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.339779 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.339819 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.339837 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 11:38:20.341869 13553 raft_consensus.cc:1261] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Refusing update from remote peer 327681b01a0846c8ab76dbedb1916c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.341931 13273 raft_consensus.cc:1261] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Refusing update from remote peer 327681b01a0846c8ab76dbedb1916c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.342278 13637 consensus_queue.cc:1035] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.342409 13635 consensus_queue.cc:1035] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.345551 13640 mvcc.cc:205] Tried to move back new op lower bound from 7298889729396436992 to 7298889729327058944. Current Snapshot: MvccSnapshot[applied={T|T < 7298889729396436992 or (T in {7298889729396436992})}]
I20260620 11:38:20.345680 13641 mvcc.cc:205] Tried to move back new op lower bound from 7298889729396436992 to 7298889729327058944. Current Snapshot: MvccSnapshot[applied={T|T < 7298889729396436992}]
I20260620 11:38:20.347374 13116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.347486 13116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.348246 13116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Sending DeleteTablet for 3 replicas of tablet 27d15f3d64e84dc997f5fc5d4ec260be
I20260620 11:38:20.348604 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d15f3d64e84dc997f5fc5d4ec260be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:42754
I20260620 11:38:20.348831 13116 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:20.348979 13116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:20.348981 13253 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d15f3d64e84dc997f5fc5d4ec260be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37802
I20260620 11:38:20.349011 13650 tablet_replica.cc:330] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: stopping tablet replica
I20260620 11:38:20.349112 13650 raft_consensus.cc:2229] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.349220 13650 raft_consensus.cc:2258] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.349198 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 27d15f3d64e84dc997f5fc5d4ec260be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:33044
I20260620 11:38:20.349671 13652 tablet_replica.cc:330] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: stopping tablet replica
I20260620 11:38:20.349790 13652 raft_consensus.cc:2229] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:38:20.349893 13652 raft_consensus.cc:2258] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.350569 13652 ts_tablet_manager.cc:1905] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.350692 13253 tablet_service.cc:1467] Processing CreateTablet for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 (DEFAULT_TABLE table=test-table [id=51a945ce9f4a4cbb8d0d158c876525b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.350807 13253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.350812 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 (DEFAULT_TABLE table=test-table [id=51a945ce9f4a4cbb8d0d158c876525b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.350924 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.352210 13630 tablet_bootstrap.cc:492] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Bootstrap starting.
I20260620 11:38:20.352310 13533 tablet_service.cc:1467] Processing CreateTablet for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 (DEFAULT_TABLE table=test-table [id=51a945ce9f4a4cbb8d0d158c876525b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.352427 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.352651 13630 tablet_bootstrap.cc:654] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.353147 13651 tablet_replica.cc:330] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: stopping tablet replica
I20260620 11:38:20.353300 13630 tablet_bootstrap.cc:492] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: No bootstrap required, opened a new log
I20260620 11:38:20.353387 13630 ts_tablet_manager.cc:1397] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:20.353447 13651 raft_consensus.cc:2229] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.353576 13651 raft_consensus.cc:2258] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.353546 13630 raft_consensus.cc:348] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.353627 13630 raft_consensus.cc:374] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.353650 13630 raft_consensus.cc:729] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69dbe66e24246e1bd20f5342952276c, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.353700 13630 consensus_queue.cc:260] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.353785 13630 ts_tablet_manager.cc:1428] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.354085 13629 tablet_bootstrap.cc:492] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: Bootstrap starting.
I20260620 11:38:20.354460 13652 ts_tablet_manager.cc:1918] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:20.354528 13629 tablet_bootstrap.cc:654] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.354562 13652 log.cc:1196] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/27d15f3d64e84dc997f5fc5d4ec260be
I20260620 11:38:20.354833 13652 ts_tablet_manager.cc:1939] T 27d15f3d64e84dc997f5fc5d4ec260be P 327681b01a0846c8ab76dbedb1916c64: Deleting consensus metadata
I20260620 11:38:20.354966 13627 tablet_bootstrap.cc:492] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Bootstrap starting.
I20260620 11:38:20.355412 13627 tablet_bootstrap.cc:654] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.355670 13651 ts_tablet_manager.cc:1905] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.356125 13627 tablet_bootstrap.cc:492] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: No bootstrap required, opened a new log
I20260620 11:38:20.356177 13627 ts_tablet_manager.cc:1397] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:20.356240 13101 catalog_manager.cc:4841] TS 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627): tablet 27d15f3d64e84dc997f5fc5d4ec260be (table test-table [id=f92ad90dd27a4ba88c7938331be5556d]) successfully deleted
I20260620 11:38:20.356321 13627 raft_consensus.cc:348] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.356389 13627 raft_consensus.cc:374] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.356410 13627 raft_consensus.cc:729] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fac61481ef94ba7a1d9860c46c31907, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.356451 13627 consensus_queue.cc:260] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.356540 13627 ts_tablet_manager.cc:1428] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.356865 13629 tablet_bootstrap.cc:492] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: No bootstrap required, opened a new log
I20260620 11:38:20.356917 13629 ts_tablet_manager.cc:1397] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260620 11:38:20.357060 13629 raft_consensus.cc:348] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.357133 13629 raft_consensus.cc:374] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.357156 13629 raft_consensus.cc:729] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.357197 13629 consensus_queue.cc:260] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.357283 13629 ts_tablet_manager.cc:1428] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 11:38:20.357710 13651 ts_tablet_manager.cc:1918] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:20.357784 13651 log.cc:1196] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/27d15f3d64e84dc997f5fc5d4ec260be
I20260620 11:38:20.357949 13650 ts_tablet_manager.cc:1905] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.358239 13651 ts_tablet_manager.cc:1939] T 27d15f3d64e84dc997f5fc5d4ec260be P e69dbe66e24246e1bd20f5342952276c: Deleting consensus metadata
I20260620 11:38:20.358667 13101 catalog_manager.cc:4841] TS e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841): tablet 27d15f3d64e84dc997f5fc5d4ec260be (table test-table [id=f92ad90dd27a4ba88c7938331be5556d]) successfully deleted
I20260620 11:38:20.359679 13650 ts_tablet_manager.cc:1918] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:20.359804 13650 log.cc:1196] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/27d15f3d64e84dc997f5fc5d4ec260be
I20260620 11:38:20.360069 13650 ts_tablet_manager.cc:1939] T 27d15f3d64e84dc997f5fc5d4ec260be P 2fac61481ef94ba7a1d9860c46c31907: Deleting consensus metadata
I20260620 11:38:20.360632 13103 catalog_manager.cc:4841] TS 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665): tablet 27d15f3d64e84dc997f5fc5d4ec260be (table test-table [id=f92ad90dd27a4ba88c7938331be5556d]) successfully deleted
W20260620 11:38:20.384617 13320 tablet.cc:2369] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:38:20.427873 13637 raft_consensus.cc:482] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:20.428018 13637 raft_consensus.cc:504] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.428268 13637 leader_election.cc:288] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841), 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665)
I20260620 11:38:20.428498 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77d1b6deb10e4c4ca74c727b78e2f7a1" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907" is_pre_election: true
I20260620 11:38:20.428511 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77d1b6deb10e4c4ca74c727b78e2f7a1" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c" is_pre_election: true
I20260620 11:38:20.428611 13273 raft_consensus.cc:2454] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 0.
I20260620 11:38:20.428611 13553 raft_consensus.cc:2454] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 0.
I20260620 11:38:20.428792 13345 leader_election.cc:302] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 327681b01a0846c8ab76dbedb1916c64, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.428915 13637 raft_consensus.cc:2781] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:20.428956 13637 raft_consensus.cc:482] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:20.428977 13637 raft_consensus.cc:3037] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.429782 13637 raft_consensus.cc:504] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.429912 13637 leader_election.cc:288] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 election: Requested vote from peers e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841), 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665)
I20260620 11:38:20.430074 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77d1b6deb10e4c4ca74c727b78e2f7a1" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c"
I20260620 11:38:20.430186 13273 raft_consensus.cc:3037] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.430151 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77d1b6deb10e4c4ca74c727b78e2f7a1" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907"
I20260620 11:38:20.430226 13553 raft_consensus.cc:3037] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.431059 13273 raft_consensus.cc:2454] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 1.
I20260620 11:38:20.431059 13553 raft_consensus.cc:2454] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 1.
I20260620 11:38:20.431267 13345 leader_election.cc:302] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 327681b01a0846c8ab76dbedb1916c64, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.431350 13637 raft_consensus.cc:2781] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:20.431411 13637 raft_consensus.cc:686] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 1 LEADER]: Becoming Leader. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Running, Role: LEADER
I20260620 11:38:20.431463 13637 consensus_queue.cc:237] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.432194 13115 catalog_manager.cc:5495] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } health_report { overall_health: UNKNOWN } } }
W20260620 11:38:20.461342 13600 tablet.cc:2369] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955499, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.190@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:20.471182 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.462856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:44709 (local address 127.0.0.1:44834)
0620 11:38:20.463032 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.463051 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.463124 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.463287 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.463288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.463636 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:20.463720 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.463722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.464713 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.464714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.465307 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.465309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.465776 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.465778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.465825 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.466512 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:20.466517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:20.468142 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:20.470057 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.470064 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.470066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.470270 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.470484 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:20.470485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:20.470485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:20.470543 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:20.470850 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:20.470857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:20.470906 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.470982 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.471089 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260620 11:38:20.471266 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.462938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.5.122.190:44709)
0620 11:38:20.462996 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.462998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.463139 (+ 141us) server_negotiation.cc:407] Connection header received
0620 11:38:20.463174 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.463175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.463187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.463272 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.463819 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.464620 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.465393 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.465701 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.465778 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.468227 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:20.468229 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:20.468229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:20.468236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:20.469920 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.470347 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.470348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.470349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.470414 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.470603 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.470604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.470604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.470747 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:20.470828 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.471025 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.471154 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":32}
I20260620 11:38:20.473472 13115 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:44834:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 11:38:20.473685 13115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:20.475273 13393 tablet_service.cc:1467] Processing CreateTablet for tablet c9ca5c16bf864ae7bc6f77c9d69ede34 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5342c216b39c4d2eb19bcc1ce44984e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:38:20.475275 13253 tablet_service.cc:1467] Processing CreateTablet for tablet c9ca5c16bf864ae7bc6f77c9d69ede34 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5342c216b39c4d2eb19bcc1ce44984e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:38:20.475293 13533 tablet_service.cc:1467] Processing CreateTablet for tablet c9ca5c16bf864ae7bc6f77c9d69ede34 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5342c216b39c4d2eb19bcc1ce44984e6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 11:38:20.475455 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ca5c16bf864ae7bc6f77c9d69ede34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.475456 13253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ca5c16bf864ae7bc6f77c9d69ede34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.475456 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ca5c16bf864ae7bc6f77c9d69ede34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.476773 13629 tablet_bootstrap.cc:492] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: Bootstrap starting.
I20260620 11:38:20.477088 13630 tablet_bootstrap.cc:492] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: Bootstrap starting.
I20260620 11:38:20.477159 13629 tablet_bootstrap.cc:654] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.477386 13627 tablet_bootstrap.cc:492] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: Bootstrap starting.
I20260620 11:38:20.477519 13630 tablet_bootstrap.cc:654] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.477864 13627 tablet_bootstrap.cc:654] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.477887 13629 tablet_bootstrap.cc:492] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: No bootstrap required, opened a new log
I20260620 11:38:20.477937 13629 ts_tablet_manager.cc:1397] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:20.478101 13629 raft_consensus.cc:348] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.478178 13629 raft_consensus.cc:374] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.478193 13629 raft_consensus.cc:729] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.478229 13630 tablet_bootstrap.cc:492] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: No bootstrap required, opened a new log
I20260620 11:38:20.478260 13629 consensus_queue.cc:260] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.478283 13630 ts_tablet_manager.cc:1397] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:20.478363 13629 ts_tablet_manager.cc:1428] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.478443 13630 raft_consensus.cc:348] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.478452 13637 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.478518 13630 raft_consensus.cc:374] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.478543 13630 raft_consensus.cc:729] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69dbe66e24246e1bd20f5342952276c, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.478536 13637 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:38:20.478595 13630 consensus_queue.cc:260] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.478721 13630 ts_tablet_manager.cc:1428] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.478801 13634 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.478902 13627 tablet_bootstrap.cc:492] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: No bootstrap required, opened a new log
I20260620 11:38:20.478907 13634 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:38:20.478955 13627 ts_tablet_manager.cc:1397] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:38:20.479113 13627 raft_consensus.cc:348] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.479202 13627 raft_consensus.cc:374] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.479224 13627 raft_consensus.cc:729] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fac61481ef94ba7a1d9860c46c31907, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.479275 13627 consensus_queue.cc:260] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.479382 13627 ts_tablet_manager.cc:1428] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260620 11:38:20.479516 13633 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.479627 13633 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:38:20.487310 13637 raft_consensus.cc:482] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:20.487421 13637 raft_consensus.cc:504] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.487596 13637 leader_election.cc:288] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841), 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665)
I20260620 11:38:20.487821 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ca5c16bf864ae7bc6f77c9d69ede34" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c" is_pre_election: true
I20260620 11:38:20.487820 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ca5c16bf864ae7bc6f77c9d69ede34" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907" is_pre_election: true
I20260620 11:38:20.487929 13553 raft_consensus.cc:2454] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 0.
I20260620 11:38:20.487929 13273 raft_consensus.cc:2454] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 0.
I20260620 11:38:20.488101 13345 leader_election.cc:302] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 327681b01a0846c8ab76dbedb1916c64, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.488199 13637 raft_consensus.cc:2781] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:20.488253 13637 raft_consensus.cc:482] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:20.488276 13637 raft_consensus.cc:3037] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.488979 13637 raft_consensus.cc:504] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.489135 13637 leader_election.cc:288] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 election: Requested vote from peers e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841), 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665)
I20260620 11:38:20.489308 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ca5c16bf864ae7bc6f77c9d69ede34" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907"
I20260620 11:38:20.489331 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ca5c16bf864ae7bc6f77c9d69ede34" candidate_uuid: "327681b01a0846c8ab76dbedb1916c64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c"
I20260620 11:38:20.489408 13273 raft_consensus.cc:3037] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.489408 13553 raft_consensus.cc:3037] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.490271 13273 raft_consensus.cc:2454] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 1.
I20260620 11:38:20.490271 13553 raft_consensus.cc:2454] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 327681b01a0846c8ab76dbedb1916c64 in term 1.
I20260620 11:38:20.490460 13347 leader_election.cc:302] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fac61481ef94ba7a1d9860c46c31907, 327681b01a0846c8ab76dbedb1916c64; no voters:
I20260620 11:38:20.490582 13637 raft_consensus.cc:2781] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:20.490665 13637 raft_consensus.cc:686] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [term 1 LEADER]: Becoming Leader. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Running, Role: LEADER
I20260620 11:38:20.490717 13637 consensus_queue.cc:237] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.491077 13645 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: TxnStatusTablet state changed. Reason: New leader 327681b01a0846c8ab76dbedb1916c64. Latest consensus state: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.491173 13645 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:38:20.491405 13667 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 11:38:20.491540 13667 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 11:38:20.491479 13116 catalog_manager.cc:5495] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } health_report { overall_health: UNKNOWN } } }
I20260620 11:38:20.502358 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.499823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33070 (local address 127.5.122.130:38627)
0620 11:38:20.499870 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.499872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.499944 (+ 72us) server_negotiation.cc:407] Connection header received
0620 11:38:20.500043 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.500044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.500056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.500129 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:20.500294 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.500798 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.501594 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.501795 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.501880 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.502072 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.502116 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.502179 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.502250 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.502275 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:38:20.502372 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.499770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:38627 (local address 127.0.0.1:33070)
0620 11:38:20.499895 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.499906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.499969 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.500138 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.500138 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.500140 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:38:20.500209 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.500212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.500894 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.500895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.501493 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.501496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.501882 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.501883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.501951 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.501990 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.502183 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.502192 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.502246 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.502303 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260620 11:38:20.503464 13273 raft_consensus.cc:1261] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Refusing update from remote peer 327681b01a0846c8ab76dbedb1916c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.503513 13553 raft_consensus.cc:1261] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Refusing update from remote peer 327681b01a0846c8ab76dbedb1916c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.503755 13645 consensus_queue.cc:1035] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.503854 13637 consensus_queue.cc:1035] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.504336 13634 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: TxnStatusTablet state changed. Reason: New leader 327681b01a0846c8ab76dbedb1916c64. Latest consensus state: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.504434 13634 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:38:20.504324 13633 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: TxnStatusTablet state changed. Reason: New leader 327681b01a0846c8ab76dbedb1916c64. Latest consensus state: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.504474 13633 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:38:20.505105 13645 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.505192 13645 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:38:20.505916 13634 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.506035 13634 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P e69dbe66e24246e1bd20f5342952276c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:38:20.506891 13635 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.506985 13635 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 327681b01a0846c8ab76dbedb1916c64: This TxnStatusTablet replica's current role is: LEADER
I20260620 11:38:20.507366 13633 tablet_replica.cc:439] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "327681b01a0846c8ab76dbedb1916c64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } } }
I20260620 11:38:20.507452 13633 tablet_replica.cc:442] T c9ca5c16bf864ae7bc6f77c9d69ede34 P 2fac61481ef94ba7a1d9860c46c31907: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 11:38:20.511989 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.509459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:58709 (local address 127.5.122.130:38627)
0620 11:38:20.509496 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.509498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.509511 (+ 13us) server_negotiation.cc:407] Connection header received
0620 11:38:20.509573 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.509574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.509582 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.509628 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:20.509763 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.510280 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.511373 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.511552 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.511615 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.511789 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.511815 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.511852 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.511904 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.511920 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 11:38:20.512069 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.509403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:38627 (local address 127.5.122.130:58709)
0620 11:38:20.509468 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.509480 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.509526 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.509651 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.509652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.509653 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:38:20.509703 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.509705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.510350 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.510351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.511276 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.511278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.511629 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.511630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.511688 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.511707 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.511853 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.511855 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.511903 (+ 48us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.511947 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 11:38:20.514330 13273 raft_consensus.cc:1261] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Refusing update from remote peer 327681b01a0846c8ab76dbedb1916c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.514330 13553 raft_consensus.cc:1261] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Refusing update from remote peer 327681b01a0846c8ab76dbedb1916c64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.514535 13645 consensus_queue.cc:1035] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.514642 13635 consensus_queue.cc:1035] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.515990 13642 mvcc.cc:205] Tried to move back new op lower bound from 7298889730104238080 to 7298889729766780928. Current Snapshot: MvccSnapshot[applied={T|T < 7298889730104238080}]
I20260620 11:38:20.525550 13115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.525681 13115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.526382 13115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Sending DeleteTablet for 3 replicas of tablet 77d1b6deb10e4c4ca74c727b78e2f7a1
I20260620 11:38:20.526660 13253 tablet_service.cc:1514] Processing DeleteTablet for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37802
I20260620 11:38:20.526736 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:42754
I20260620 11:38:20.526795 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:33044
I20260620 11:38:20.526858 13651 tablet_replica.cc:330] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: stopping tablet replica
I20260620 11:38:20.526921 13651 raft_consensus.cc:2229] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.526955 13651 raft_consensus.cc:2258] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.527050 13650 tablet_replica.cc:330] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: stopping tablet replica
I20260620 11:38:20.527127 13650 raft_consensus.cc:2229] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.527168 13650 raft_consensus.cc:2258] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.527256 13652 tablet_replica.cc:330] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: stopping tablet replica
I20260620 11:38:20.527316 13652 raft_consensus.cc:2229] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:38:20.527387 13652 raft_consensus.cc:2258] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.527772 13650 ts_tablet_manager.cc:1905] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.529155 13652 ts_tablet_manager.cc:1905] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.529335 13650 ts_tablet_manager.cc:1918] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:38:20.529397 13650 log.cc:1196] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/77d1b6deb10e4c4ca74c727b78e2f7a1
I20260620 11:38:20.529626 13650 ts_tablet_manager.cc:1939] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 2fac61481ef94ba7a1d9860c46c31907: Deleting consensus metadata
I20260620 11:38:20.529982 13103 catalog_manager.cc:4841] TS 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665): tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 (table test-table [id=51a945ce9f4a4cbb8d0d158c876525b7]) successfully deleted
I20260620 11:38:20.530776 13652 ts_tablet_manager.cc:1918] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:38:20.530834 13652 log.cc:1196] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/77d1b6deb10e4c4ca74c727b78e2f7a1
I20260620 11:38:20.531041 13652 ts_tablet_manager.cc:1939] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P 327681b01a0846c8ab76dbedb1916c64: Deleting consensus metadata
I20260620 11:38:20.531387 13101 catalog_manager.cc:4841] TS 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627): tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 (table test-table [id=51a945ce9f4a4cbb8d0d158c876525b7]) successfully deleted
I20260620 11:38:20.533957 13651 ts_tablet_manager.cc:1905] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.535904 13651 ts_tablet_manager.cc:1918] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:38:20.535977 13651 log.cc:1196] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/77d1b6deb10e4c4ca74c727b78e2f7a1
I20260620 11:38:20.536212 13651 ts_tablet_manager.cc:1939] T 77d1b6deb10e4c4ca74c727b78e2f7a1 P e69dbe66e24246e1bd20f5342952276c: Deleting consensus metadata
I20260620 11:38:20.536590 13101 catalog_manager.cc:4841] TS e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841): tablet 77d1b6deb10e4c4ca74c727b78e2f7a1 (table test-table [id=51a945ce9f4a4cbb8d0d158c876525b7]) successfully deleted
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 11:38:20.555881 13165 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:20.549535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.5.122.190:44709)
0620 11:38:20.549592 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.549594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.549704 (+ 110us) server_negotiation.cc:407] Connection header received
0620 11:38:20.549773 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.549774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.549784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.549837 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.550605 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.551091 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.552216 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.552407 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.552462 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.555788 (+ 3326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44840: BlockingRecv error: recv got EOF from 127.0.0.1:44840 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13071](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM
I20260620 11:38:20.565034 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.556872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.5.122.190:44709)
0620 11:38:20.556928 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.556930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.557012 (+ 82us) server_negotiation.cc:407] Connection header received
0620 11:38:20.557080 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.557081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.557092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.557146 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:20.557628 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.558397 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.559335 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.559609 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.559683 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.562182 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:20.562184 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:20.562184 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:20.562193 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:20.563745 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.564138 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.564139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.564140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.564210 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:20.564447 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:20.564448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:20.564449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:20.564591 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:20.564698 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.564857 (+ 159us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.564940 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 11:38:20.567023 13115 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:20.567219 13115 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:20.568838 13533 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc126b403b24fbf8b6f3e345abe3500 (DEFAULT_TABLE table=test-table [id=d440cdaef4134edb9798ca197190466e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.568908 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc126b403b24fbf8b6f3e345abe3500 (DEFAULT_TABLE table=test-table [id=d440cdaef4134edb9798ca197190466e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.568928 13253 tablet_service.cc:1467] Processing CreateTablet for tablet 0dc126b403b24fbf8b6f3e345abe3500 (DEFAULT_TABLE table=test-table [id=d440cdaef4134edb9798ca197190466e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.568990 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc126b403b24fbf8b6f3e345abe3500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.569046 13253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc126b403b24fbf8b6f3e345abe3500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.569046 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dc126b403b24fbf8b6f3e345abe3500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.570369 13627 tablet_bootstrap.cc:492] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: Bootstrap starting.
I20260620 11:38:20.570439 13629 tablet_bootstrap.cc:492] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: Bootstrap starting.
I20260620 11:38:20.570698 13629 tablet_bootstrap.cc:654] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.570986 13630 tablet_bootstrap.cc:492] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: Bootstrap starting.
I20260620 11:38:20.571038 13627 tablet_bootstrap.cc:654] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.571394 13630 tablet_bootstrap.cc:654] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.571723 13627 tablet_bootstrap.cc:492] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: No bootstrap required, opened a new log
I20260620 11:38:20.571766 13627 ts_tablet_manager.cc:1397] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:20.571929 13627 raft_consensus.cc:348] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.572005 13627 raft_consensus.cc:374] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.572026 13627 raft_consensus.cc:729] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fac61481ef94ba7a1d9860c46c31907, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.572069 13627 consensus_queue.cc:260] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.572186 13627 ts_tablet_manager.cc:1428] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.572474 13629 tablet_bootstrap.cc:492] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: No bootstrap required, opened a new log
I20260620 11:38:20.572552 13629 ts_tablet_manager.cc:1397] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:38:20.572674 13629 raft_consensus.cc:348] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.572755 13629 raft_consensus.cc:374] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.572788 13629 raft_consensus.cc:729] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.572841 13629 consensus_queue.cc:260] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.572937 13629 ts_tablet_manager.cc:1428] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.573438 13630 tablet_bootstrap.cc:492] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: No bootstrap required, opened a new log
I20260620 11:38:20.573511 13630 ts_tablet_manager.cc:1397] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260620 11:38:20.573673 13630 raft_consensus.cc:348] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.573750 13630 raft_consensus.cc:374] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.573776 13630 raft_consensus.cc:729] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69dbe66e24246e1bd20f5342952276c, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.573819 13630 consensus_queue.cc:260] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.573928 13630 ts_tablet_manager.cc:1428] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.699903 13633 raft_consensus.cc:482] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:20.699998 13633 raft_consensus.cc:504] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.700316 13633 leader_election.cc:288] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841), 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627)
I20260620 11:38:20.703208 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.700479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:45841 (local address 127.5.122.131:42817)
0620 11:38:20.700539 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.700552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.700638 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.700786 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.700787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.700789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.700883 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.700886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.701585 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.701586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.702461 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.702463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.702995 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.702996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.703072 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.703076 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.703127 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.703154 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260620 11:38:20.703377 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.700591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:42817 (local address 127.5.122.129:45841)
0620 11:38:20.700648 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.700650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.700664 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:38:20.700697 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.700698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.700707 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.700771 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.700976 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.701508 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.702543 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.702919 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.702975 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.702981 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.703250 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.703307 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 11:38:20.703459 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.700721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:45941 (local address 127.5.122.130:38627)
0620 11:38:20.700797 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.700799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.700836 (+ 37us) server_negotiation.cc:407] Connection header received
0620 11:38:20.700902 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.700903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.700912 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.700968 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.701120 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.701603 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.702595 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.702940 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.703031 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.703041 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.703349 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.703389 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 11:38:20.703567 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc126b403b24fbf8b6f3e345abe3500" candidate_uuid: "2fac61481ef94ba7a1d9860c46c31907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c" is_pre_election: true
I20260620 11:38:20.703655 13273 raft_consensus.cc:2454] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fac61481ef94ba7a1d9860c46c31907 in term 0.
I20260620 11:38:20.703675 13691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.700585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:38627 (local address 127.5.122.131:45941)
0620 11:38:20.700786 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.700795 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.700849 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.700980 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.700981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.700983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.701056 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.701058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.701694 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.701695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.702495 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.702514 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.703054 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.703055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.703251 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.703256 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.703558 (+ 302us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.703601 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":62,"threads_started":1}
I20260620 11:38:20.703881 13485 leader_election.cc:302] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fac61481ef94ba7a1d9860c46c31907, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.703928 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc126b403b24fbf8b6f3e345abe3500" candidate_uuid: "2fac61481ef94ba7a1d9860c46c31907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "327681b01a0846c8ab76dbedb1916c64" is_pre_election: true
I20260620 11:38:20.704036 13633 raft_consensus.cc:2781] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:20.704077 13633 raft_consensus.cc:482] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:20.704088 13413 raft_consensus.cc:2454] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fac61481ef94ba7a1d9860c46c31907 in term 0.
I20260620 11:38:20.704107 13633 raft_consensus.cc:3037] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.704806 13633 raft_consensus.cc:504] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.704908 13633 leader_election.cc:288] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [CANDIDATE]: Term 1 election: Requested vote from peers e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841), 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627)
I20260620 11:38:20.705091 13273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc126b403b24fbf8b6f3e345abe3500" candidate_uuid: "2fac61481ef94ba7a1d9860c46c31907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69dbe66e24246e1bd20f5342952276c"
I20260620 11:38:20.705097 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dc126b403b24fbf8b6f3e345abe3500" candidate_uuid: "2fac61481ef94ba7a1d9860c46c31907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "327681b01a0846c8ab76dbedb1916c64"
I20260620 11:38:20.705168 13413 raft_consensus.cc:3037] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.705184 13273 raft_consensus.cc:3037] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.705777 13413 raft_consensus.cc:2454] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fac61481ef94ba7a1d9860c46c31907 in term 1.
I20260620 11:38:20.705855 13273 raft_consensus.cc:2454] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fac61481ef94ba7a1d9860c46c31907 in term 1.
I20260620 11:38:20.705952 13485 leader_election.cc:302] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fac61481ef94ba7a1d9860c46c31907, 327681b01a0846c8ab76dbedb1916c64; no voters:
I20260620 11:38:20.706110 13633 raft_consensus.cc:2781] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:20.706230 13633 raft_consensus.cc:686] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 LEADER]: Becoming Leader. State: Replica: 2fac61481ef94ba7a1d9860c46c31907, State: Running, Role: LEADER
I20260620 11:38:20.706302 13633 consensus_queue.cc:237] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.707095 13116 catalog_manager.cc:5495] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "2fac61481ef94ba7a1d9860c46c31907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } health_report { overall_health: HEALTHY } } }
I20260620 11:38:20.723424 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.720807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.5.122.131:46665)
0620 11:38:20.720875 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.720877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.720887 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:20.720976 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.720979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.721003 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.721101 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:20.721210 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.721741 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.722576 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.722772 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.722852 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.723048 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.723159 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.723241 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.723294 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.723315 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:38:20.724505 13273 raft_consensus.cc:1261] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Refusing update from remote peer 2fac61481ef94ba7a1d9860c46c31907: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.724574 13413 raft_consensus.cc:1261] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Refusing update from remote peer 2fac61481ef94ba7a1d9860c46c31907: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.724792 13633 consensus_queue.cc:1035] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.724977 13692 consensus_queue.cc:1035] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.728646 13114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.728763 13114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.729460 13114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Sending DeleteTablet for 3 replicas of tablet 0dc126b403b24fbf8b6f3e345abe3500
I20260620 11:38:20.729737 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dc126b403b24fbf8b6f3e345abe3500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:33044
I20260620 11:38:20.729790 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dc126b403b24fbf8b6f3e345abe3500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:42754
I20260620 11:38:20.729836 13652 tablet_replica.cc:330] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: stopping tablet replica
I20260620 11:38:20.729801 13253 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dc126b403b24fbf8b6f3e345abe3500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37802
I20260620 11:38:20.729871 13650 tablet_replica.cc:330] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: stopping tablet replica
I20260620 11:38:20.729900 13652 raft_consensus.cc:2229] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.729907 13651 tablet_replica.cc:330] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: stopping tablet replica
I20260620 11:38:20.729911 13650 raft_consensus.cc:2229] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:38:20.729945 13652 raft_consensus.cc:2258] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.729985 13651 raft_consensus.cc:2229] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.730034 13651 raft_consensus.cc:2258] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.730037 13650 raft_consensus.cc:2258] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.730345 13650 ts_tablet_manager.cc:1905] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.730438 13651 ts_tablet_manager.cc:1905] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.730629 13114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:20.730772 13114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:20.732241 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 064f7571bd174213823c3a0c0ca88fb3 (DEFAULT_TABLE table=test-table [id=d0187c37abd645c1be2a7dafffc2362a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.732334 13533 tablet_service.cc:1467] Processing CreateTablet for tablet 064f7571bd174213823c3a0c0ca88fb3 (DEFAULT_TABLE table=test-table [id=d0187c37abd645c1be2a7dafffc2362a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.732364 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f7571bd174213823c3a0c0ca88fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.732564 13253 tablet_service.cc:1467] Processing CreateTablet for tablet 064f7571bd174213823c3a0c0ca88fb3 (DEFAULT_TABLE table=test-table [id=d0187c37abd645c1be2a7dafffc2362a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:20.732713 13651 ts_tablet_manager.cc:1918] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:20.732755 13651 log.cc:1196] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/0dc126b403b24fbf8b6f3e345abe3500
I20260620 11:38:20.732982 13651 ts_tablet_manager.cc:1939] T 0dc126b403b24fbf8b6f3e345abe3500 P e69dbe66e24246e1bd20f5342952276c: Deleting consensus metadata
I20260620 11:38:20.733111 13253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f7571bd174213823c3a0c0ca88fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.733834 13652 ts_tablet_manager.cc:1905] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.734342 13629 tablet_bootstrap.cc:492] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: Bootstrap starting.
I20260620 11:38:20.734685 13629 tablet_bootstrap.cc:654] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.734938 13630 tablet_bootstrap.cc:492] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: Bootstrap starting.
I20260620 11:38:20.735095 13652 ts_tablet_manager.cc:1918] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:20.735142 13652 log.cc:1196] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/0dc126b403b24fbf8b6f3e345abe3500
I20260620 11:38:20.735311 13652 ts_tablet_manager.cc:1939] T 0dc126b403b24fbf8b6f3e345abe3500 P 327681b01a0846c8ab76dbedb1916c64: Deleting consensus metadata
I20260620 11:38:20.735592 13630 tablet_bootstrap.cc:654] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.736264 13629 tablet_bootstrap.cc:492] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: No bootstrap required, opened a new log
I20260620 11:38:20.736331 13629 ts_tablet_manager.cc:1397] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260620 11:38:20.736378 13101 catalog_manager.cc:4841] TS 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627): tablet 0dc126b403b24fbf8b6f3e345abe3500 (table test-table [id=d440cdaef4134edb9798ca197190466e]) successfully deleted
I20260620 11:38:20.736444 13629 raft_consensus.cc:348] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.736518 13629 raft_consensus.cc:374] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.736541 13629 raft_consensus.cc:729] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 327681b01a0846c8ab76dbedb1916c64, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.736593 13629 consensus_queue.cc:260] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.736685 13629 ts_tablet_manager.cc:1428] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.736950 13630 tablet_bootstrap.cc:492] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: No bootstrap required, opened a new log
I20260620 11:38:20.736999 13630 ts_tablet_manager.cc:1397] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:38:20.737150 13630 raft_consensus.cc:348] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.737221 13630 raft_consensus.cc:374] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.737241 13630 raft_consensus.cc:729] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69dbe66e24246e1bd20f5342952276c, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.737290 13630 consensus_queue.cc:260] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.737393 13630 ts_tablet_manager.cc:1428] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.737859 13650 ts_tablet_manager.cc:1918] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:20.737923 13650 log.cc:1196] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/0dc126b403b24fbf8b6f3e345abe3500
I20260620 11:38:20.738178 13650 ts_tablet_manager.cc:1939] T 0dc126b403b24fbf8b6f3e345abe3500 P 2fac61481ef94ba7a1d9860c46c31907: Deleting consensus metadata
I20260620 11:38:20.738421 13533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064f7571bd174213823c3a0c0ca88fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:20.739753 13627 tablet_bootstrap.cc:492] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: Bootstrap starting.
I20260620 11:38:20.740238 13627 tablet_bootstrap.cc:654] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:20.740288 13101 catalog_manager.cc:4841] TS e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841): tablet 0dc126b403b24fbf8b6f3e345abe3500 (table test-table [id=d440cdaef4134edb9798ca197190466e]) successfully deleted
I20260620 11:38:20.740622 13103 catalog_manager.cc:4841] TS 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665): tablet 0dc126b403b24fbf8b6f3e345abe3500 (table test-table [id=d440cdaef4134edb9798ca197190466e]) successfully deleted
I20260620 11:38:20.740922 13627 tablet_bootstrap.cc:492] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: No bootstrap required, opened a new log
I20260620 11:38:20.740964 13627 ts_tablet_manager.cc:1397] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:20.741079 13627 raft_consensus.cc:348] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.741138 13627 raft_consensus.cc:374] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:20.741153 13627 raft_consensus.cc:729] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fac61481ef94ba7a1d9860c46c31907, State: Initialized, Role: FOLLOWER
I20260620 11:38:20.741216 13627 consensus_queue.cc:260] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.741341 13627 ts_tablet_manager.cc:1428] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:20.771744 13634 raft_consensus.cc:482] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:20.771858 13634 raft_consensus.cc:504] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.772234 13634 leader_election.cc:288] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627), 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665)
I20260620 11:38:20.776079 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.772475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:45801 (local address 127.5.122.131:46665)
0620 11:38:20.772521 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.772524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.772660 (+ 136us) server_negotiation.cc:407] Connection header received
0620 11:38:20.772771 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.772772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.772785 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.772839 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.773030 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.773497 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.775117 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.775720 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.775809 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.775818 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.775964 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.776003 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:38:20.776099 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.772413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:46665 (local address 127.5.122.129:45801)
0620 11:38:20.772600 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.772609 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.772685 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.772882 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.772882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.772884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.772937 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.772938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.773618 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.773619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.775007 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.775011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.775814 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.775815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.775874 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.775878 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.775953 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.776001 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":98,"thread_start_us":54,"threads_started":1}
I20260620 11:38:20.776666 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f7571bd174213823c3a0c0ca88fb3" candidate_uuid: "e69dbe66e24246e1bd20f5342952276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907" is_pre_election: true
I20260620 11:38:20.776784 13553 raft_consensus.cc:2454] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69dbe66e24246e1bd20f5342952276c in term 0.
I20260620 11:38:20.776850 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.772591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:40899 (local address 127.5.122.130:38627)
0620 11:38:20.772638 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.772640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.772649 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:20.772713 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.772713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.772724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.772773 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:20.773002 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.773759 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.775865 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.776449 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.776529 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.776537 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.776741 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.776789 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:38:20.776858 13702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.772413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:38627 (local address 127.5.122.129:40899)
0620 11:38:20.772564 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.772575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.772639 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.772797 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.772798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.772800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:20.772909 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.772912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.773852 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.773853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.775008 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.775011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.776547 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.776548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.776651 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.776655 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.776731 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.776772 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":5,"thread_start_us":68,"threads_started":1}
I20260620 11:38:20.777122 13207 leader_election.cc:302] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fac61481ef94ba7a1d9860c46c31907, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.777138 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f7571bd174213823c3a0c0ca88fb3" candidate_uuid: "e69dbe66e24246e1bd20f5342952276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "327681b01a0846c8ab76dbedb1916c64" is_pre_election: true
I20260620 11:38:20.777231 13413 raft_consensus.cc:2454] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69dbe66e24246e1bd20f5342952276c in term 0.
I20260620 11:38:20.777283 13634 raft_consensus.cc:2781] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:20.777334 13634 raft_consensus.cc:482] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:20.777356 13634 raft_consensus.cc:3037] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.778429 13634 raft_consensus.cc:504] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.778586 13634 leader_election.cc:288] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [CANDIDATE]: Term 1 election: Requested vote from peers 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627), 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665)
I20260620 11:38:20.778822 13413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f7571bd174213823c3a0c0ca88fb3" candidate_uuid: "e69dbe66e24246e1bd20f5342952276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "327681b01a0846c8ab76dbedb1916c64"
I20260620 11:38:20.778858 13553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064f7571bd174213823c3a0c0ca88fb3" candidate_uuid: "e69dbe66e24246e1bd20f5342952276c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fac61481ef94ba7a1d9860c46c31907"
I20260620 11:38:20.778937 13413 raft_consensus.cc:3037] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.778975 13553 raft_consensus.cc:3037] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:20.779599 13553 raft_consensus.cc:2454] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69dbe66e24246e1bd20f5342952276c in term 1.
I20260620 11:38:20.779796 13207 leader_election.cc:302] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fac61481ef94ba7a1d9860c46c31907, e69dbe66e24246e1bd20f5342952276c; no voters:
I20260620 11:38:20.779844 13413 raft_consensus.cc:2454] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69dbe66e24246e1bd20f5342952276c in term 1.
I20260620 11:38:20.780030 13634 raft_consensus.cc:2781] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:20.780193 13634 raft_consensus.cc:686] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 1 LEADER]: Becoming Leader. State: Replica: e69dbe66e24246e1bd20f5342952276c, State: Running, Role: LEADER
I20260620 11:38:20.780277 13634 consensus_queue.cc:237] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } }
I20260620 11:38:20.781100 13114 catalog_manager.cc:5495] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c reported cstate change: term changed from 0 to 1, leader changed from <none> to e69dbe66e24246e1bd20f5342952276c (127.5.122.129). New cstate: current_term: 1 leader_uuid: "e69dbe66e24246e1bd20f5342952276c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e69dbe66e24246e1bd20f5342952276c" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 } health_report { overall_health: UNKNOWN } } }
I20260620 11:38:20.800062 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.797384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37812 (local address 127.5.122.129:45841)
0620 11:38:20.797430 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.797432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.797441 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:20.797539 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.797540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.797567 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.797679 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:20.797828 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.798413 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.799188 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.799424 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.799507 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.799685 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.799805 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.799906 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.799946 (+ 40us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.799967 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:38:20.805660 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.803039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:59645 (local address 127.5.122.130:38627)
0620 11:38:20.803093 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.803095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.803195 (+ 100us) server_negotiation.cc:407] Connection header received
0620 11:38:20.803265 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.803265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.803276 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.803332 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:20.803496 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.804047 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.804854 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.805062 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.805142 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.805369 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.805410 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.805464 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.805547 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.805577 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 11:38:20.805706 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.802957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:38627 (local address 127.5.122.129:59645)
0620 11:38:20.803119 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.803130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.803199 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.803335 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.803335 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.803337 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:38:20.803412 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.803413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.804136 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.804137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.804759 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.804761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.805165 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.805166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.805236 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.805252 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.805495 (+ 243us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.805497 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.805546 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.805607 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260620 11:38:20.808262 13553 raft_consensus.cc:1261] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Refusing update from remote peer e69dbe66e24246e1bd20f5342952276c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.808537 13413 raft_consensus.cc:1261] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Refusing update from remote peer e69dbe66e24246e1bd20f5342952276c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:20.808593 13704 consensus_queue.cc:1035] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fac61481ef94ba7a1d9860c46c31907" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.808773 13634 consensus_queue.cc:1035] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [LEADER]: Connected to new peer: Peer: permanent_uuid: "327681b01a0846c8ab76dbedb1916c64" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:20.817391 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.814419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:45841 (local address 127.5.122.130:58245)
0620 11:38:20.814467 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:20.814479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:20.814518 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:20.814703 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:20.814704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:20.814706 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 11:38:20.814756 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.814758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.815458 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.815459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.816326 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:20.816330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.816840 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.816841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:20.816899 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.816906 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.817155 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.817158 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:20.817275 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 11:38:20.817328 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260620 11:38:20.817404 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:20.814453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:58245 (local address 127.5.122.129:45841)
0620 11:38:20.814497 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:20.814499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:20.814508 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:20.814585 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:20.814586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:20.814597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:20.814653 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:20.814853 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.815353 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.816473 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:20.816752 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:20.816807 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:20.817003 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:20.817043 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:20.817100 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:20.817286 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 11:38:20.817312 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:38:20.826970 13115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.827119 13115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:20.827836 13115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dafcaffd92c74db3bd67ce19548854ae: Sending DeleteTablet for 3 replicas of tablet 064f7571bd174213823c3a0c0ca88fb3
I20260620 11:38:20.828188 13533 tablet_service.cc:1514] Processing DeleteTablet for tablet 064f7571bd174213823c3a0c0ca88fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:42754
I20260620 11:38:20.828329 13650 tablet_replica.cc:330] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: stopping tablet replica
I20260620 11:38:20.828384 13650 raft_consensus.cc:2229] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.828423 13650 raft_consensus.cc:2258] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.828559 13253 tablet_service.cc:1514] Processing DeleteTablet for tablet 064f7571bd174213823c3a0c0ca88fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37802
I20260620 11:38:20.828655 13651 tablet_replica.cc:330] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: stopping tablet replica
I20260620 11:38:20.828704 13651 raft_consensus.cc:2229] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:38:20.828797 13651 raft_consensus.cc:2258] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.829015 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 064f7571bd174213823c3a0c0ca88fb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:20 UTC) from {username='oryx', principal='oryx/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:33044
I20260620 11:38:20.829111 13652 tablet_replica.cc:330] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: stopping tablet replica
I20260620 11:38:20.829156 13652 raft_consensus.cc:2229] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:20.829193 13652 raft_consensus.cc:2258] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:20.829211 13651 ts_tablet_manager.cc:1905] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.829576 13652 ts_tablet_manager.cc:1905] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.831125 13651 ts_tablet_manager.cc:1918] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:38:20.831189 13651 log.cc:1196] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/064f7571bd174213823c3a0c0ca88fb3
I20260620 11:38:20.831171 13652 ts_tablet_manager.cc:1918] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:38:20.831262 13652 log.cc:1196] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/064f7571bd174213823c3a0c0ca88fb3
I20260620 11:38:20.831426 13651 ts_tablet_manager.cc:1939] T 064f7571bd174213823c3a0c0ca88fb3 P e69dbe66e24246e1bd20f5342952276c: Deleting consensus metadata
I20260620 11:38:20.831485 13652 ts_tablet_manager.cc:1939] T 064f7571bd174213823c3a0c0ca88fb3 P 327681b01a0846c8ab76dbedb1916c64: Deleting consensus metadata
I20260620 11:38:20.831826 13101 catalog_manager.cc:4841] TS e69dbe66e24246e1bd20f5342952276c (127.5.122.129:45841): tablet 064f7571bd174213823c3a0c0ca88fb3 (table test-table [id=d0187c37abd645c1be2a7dafffc2362a]) successfully deleted
I20260620 11:38:20.832188 13101 catalog_manager.cc:4841] TS 327681b01a0846c8ab76dbedb1916c64 (127.5.122.130:38627): tablet 064f7571bd174213823c3a0c0ca88fb3 (table test-table [id=d0187c37abd645c1be2a7dafffc2362a]) successfully deleted
I20260620 11:38:20.832262 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 13188
I20260620 11:38:20.834432 13650 ts_tablet_manager.cc:1905] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:20.836277 13650 ts_tablet_manager.cc:1918] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 11:38:20.836365 13650 log.cc:1196] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/064f7571bd174213823c3a0c0ca88fb3
I20260620 11:38:20.836580 13650 ts_tablet_manager.cc:1939] T 064f7571bd174213823c3a0c0ca88fb3 P 2fac61481ef94ba7a1d9860c46c31907: Deleting consensus metadata
I20260620 11:38:20.837162 13103 catalog_manager.cc:4841] TS 2fac61481ef94ba7a1d9860c46c31907 (127.5.122.131:46665): tablet 064f7571bd174213823c3a0c0ca88fb3 (table test-table [id=d0187c37abd645c1be2a7dafffc2362a]) successfully deleted
I20260620 11:38:20.839673 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 13328
I20260620 11:38:20.848124 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 13468
I20260620 11:38:20.853771 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 13084
2026-06-20T11:38:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2479 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13715](info): set up 2 sockets
Jun 20 11:38:20 dist-test-slave-szdr krb5kdc[13715](info): commencing operation
krb5kdc: starting...
W20260620 11:38:21.898723 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:21 dist-test-slave-szdr krb5kdc[13715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:22.025107 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41285
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:22.110363 13728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:22.110618 13728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:22.110657 13728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:22.111860 13728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:22.111929 13728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:22.111963 13728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:22.111991 13728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:22.112015 13728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:22.113572 13728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:22.113862 13728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:22.114140 13728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:22.119310 13736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:22.119506 13728 server_base.cc:1034] running on GCE node
W20260620 11:38:22.119412 13734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:22.119603 13733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:22.119867 13728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:22.120173 13728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:22.121366 13728 hybrid_clock.cc:648] HybridClock initialized: now 1781955502121353 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T11:38:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1267 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:22 dist-test-slave-szdr krb5kdc[13741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:22 dist-test-slave-szdr krb5kdc[13741](info): set up 2 sockets
Jun 20 11:38:22 dist-test-slave-szdr krb5kdc[13741](info): commencing operation
krb5kdc: starting...
W20260620 11:38:23.166666 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:23.291298 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38593
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:38593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:23.372056 13754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:23.372308 13754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:23.372344 13754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:23.373497 13754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:23.373574 13754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:23.373597 13754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:23.373632 13754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:23.373656 13754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:23.375200 13754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:38593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:23.375485 13754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:23.375723 13754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:23.380833 13759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:23.380833 13762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:23.380836 13760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:23.380916 13754 server_base.cc:1034] running on GCE node
I20260620 11:38:23.381364 13754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:23.381654 13754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:23.382803 13754 hybrid_clock.cc:648] HybridClock initialized: now 1781955503382787 us; error 28 us; skew 500 ppm
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:23.390111 13754 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:23.390686 13754 webserver.cc:460] Webserver started at http://127.5.122.190:45231/ using document root <none> and password file <none>
I20260620 11:38:23.390938 13754 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:23.391012 13754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:23.391180 13754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:23.392330 13754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "0663b82c2be24bb988313ef6725af425"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "92ca37b14185fec8398d4c15874ade9b"
server_key_iv: "d03aa8bd35ba03da4d8c808f0aa8dbd1"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.392724 13754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "0663b82c2be24bb988313ef6725af425"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "92ca37b14185fec8398d4c15874ade9b"
server_key_iv: "d03aa8bd35ba03da4d8c808f0aa8dbd1"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.394254 13754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:23.395057 13769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.395255 13754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:23.395367 13754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "0663b82c2be24bb988313ef6725af425"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "92ca37b14185fec8398d4c15874ade9b"
server_key_iv: "d03aa8bd35ba03da4d8c808f0aa8dbd1"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.395450 13754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:23.411542 13754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:23.413697 13754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:23.413913 13754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:23.418449 13754 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:38593
I20260620 11:38:23.418485 13821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:38593 every 8 connection(s)
I20260620 11:38:23.418867 13754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:23.419874 13822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:23.422382 13822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Bootstrap starting.
I20260620 11:38:23.423013 13822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:23.423331 13822 log.cc:824] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:23.424176 13822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: No bootstrap required, opened a new log
I20260620 11:38:23.425828 13822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:23.426010 13822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:23.426059 13822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0663b82c2be24bb988313ef6725af425, State: Initialized, Role: FOLLOWER
I20260620 11:38:23.426232 13822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:23.426297 13822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:23.426340 13822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:23.426391 13822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:23.426599 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 13754
I20260620 11:38:23.426735 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:23.426923 5610 external_mini_cluster.cc:1428] Setting key b8e01d9b6bafd4e213a7663fad60f4b1
I20260620 11:38:23.427160 13822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:23.427286 13822 leader_election.cc:302] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0663b82c2be24bb988313ef6725af425; no voters:
I20260620 11:38:23.427469 13822 leader_election.cc:288] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:23.427524 13826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:23.427801 13826 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 LEADER]: Becoming Leader. State: Replica: 0663b82c2be24bb988313ef6725af425, State: Running, Role: LEADER
I20260620 11:38:23.427848 13822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:23.427955 13826 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:23.428421 13829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0663b82c2be24bb988313ef6725af425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } } }
I20260620 11:38:23.428666 13829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:23.428586 13827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0663b82c2be24bb988313ef6725af425. Latest consensus state: current_term: 1 leader_uuid: "0663b82c2be24bb988313ef6725af425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } } }
I20260620 11:38:23.428941 13827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:23.429173 13833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:23.429710 13833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:23.431623 13833 catalog_manager.cc:1347] Generated new cluster ID: b2222abf34dc41c9b45ea124409d60cd
I20260620 11:38:23.431684 13833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:23.439260 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:23.427611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.5.122.190:38593)
0620 11:38:23.427773 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:23.427784 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:23.427826 (+ 42us) server_negotiation.cc:407] Connection header received
0620 11:38:23.427982 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:23.427990 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:23.428056 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:23.428240 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:23.429130 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.430550 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.431579 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.432191 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.432297 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:23.435398 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:23.435402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:23.435403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:23.435416 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:23.437421 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.437877 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.437880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.437881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.438047 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.438269 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.438270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.438270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.438588 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:23.438735 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:23.438864 (+ 129us) server_negotiation.cc:299] Negotiation successful
0620 11:38:23.439018 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260620 11:38:23.444661 13833 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:23.445411 13833 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:23.451886 13833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Generated new TSK 0
I20260620 11:38:23.452134 13833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:38:23.485303 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38593
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:23.569538 13848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:23.569834 13848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:23.569870 13848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:23.571161 13848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:23.571209 13848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:23.571237 13848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:23.572685 13848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:23.572896 13848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:23.573081 13848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:23.578231 13856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:23.578291 13854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:23.578318 13848 server_base.cc:1034] running on GCE node
W20260620 11:38:23.578231 13853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:23.578747 13848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:23.579027 13848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:23.580228 13848 hybrid_clock.cc:648] HybridClock initialized: now 1781955503580210 us; error 32 us; skew 500 ppm
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:23.583007 13848 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:38:23.583470 13848 webserver.cc:460] Webserver started at http://127.5.122.129:34733/ using document root <none> and password file <none>
I20260620 11:38:23.583727 13848 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:23.583798 13848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:23.583943 13848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:23.585065 13848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "463cb02dd09d400da3ae38433249801f"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "dcbb0675ba3eed4848612860a47acd1a"
server_key_iv: "7900eb248cb58aea10798dc665dc6ab8"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.585438 13848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "463cb02dd09d400da3ae38433249801f"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "dcbb0675ba3eed4848612860a47acd1a"
server_key_iv: "7900eb248cb58aea10798dc665dc6ab8"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.586863 13848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:23.587630 13863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.587827 13848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.587931 13848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "463cb02dd09d400da3ae38433249801f"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "dcbb0675ba3eed4848612860a47acd1a"
server_key_iv: "7900eb248cb58aea10798dc665dc6ab8"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.588006 13848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:23.602401 13848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:23.604993 13848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:23.605183 13848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:23.605466 13848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:23.605811 13848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:23.605866 13848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.605901 13848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:23.605940 13848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.612092 13848 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:44043
I20260620 11:38:23.612139 13976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:44043 every 8 connection(s)
I20260620 11:38:23.612489 13848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:23.620661 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 13848
I20260620 11:38:23.620816 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:23.621028 5610 external_mini_cluster.cc:1428] Setting key f6912c5f9014c762624b024a8e50e730
I20260620 11:38:23.623193 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:23.613287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:52673 (local address 127.5.122.190:38593)
0620 11:38:23.613369 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:23.613371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:23.613593 (+ 222us) server_negotiation.cc:407] Connection header received
0620 11:38:23.614150 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:23.614151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:23.614165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:23.614234 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:23.615253 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.615773 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.616821 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.617085 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.617144 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:23.619593 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:23.619595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:23.619595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:23.619604 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:23.621419 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.621995 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.621997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.621997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.622078 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.622454 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.622456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.622456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.622586 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:23.622652 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:23.623028 (+ 376us) server_negotiation.cc:299] Negotiation successful
0620 11:38:23.623117 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 11:38:23.623518 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:23.613273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:52673)
0620 11:38:23.613451 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:23.613495 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:23.614053 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:23.614324 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:23.614328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:23.614931 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:23.615148 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:23.615154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.615865 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.615866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:23.616718 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:23.616720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.617169 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.617169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:23.617307 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:23.617939 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:23.617950 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:23.619485 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:23.621550 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:23.621561 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:23.621564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:23.621885 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:23.622192 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:23.622192 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:23.622193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:23.622365 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:23.622663 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:23.622672 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:23.622879 (+ 207us) client_negotiation.cc:769] Sending connection context
0620 11:38:23.623001 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 11:38:23.623213 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260620 11:38:23.624394 13977 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38593
I20260620 11:38:23.624529 13977 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:23.624787 13977 heartbeater.cc:511] Master 127.5.122.190:38593 requested a full tablet report, sending...
I20260620 11:38:23.625432 13786 ts_manager.cc:194] Registered new tserver with Master: 463cb02dd09d400da3ae38433249801f (127.5.122.129:44043)
I20260620 11:38:23.626448 13786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:52673
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:38:23.649611 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38593
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:23.732798 13982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:23.733062 13982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:23.733101 13982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:23.734376 13982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:23.734452 13982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:23.734495 13982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:23.736033 13982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:23.736325 13982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:23.736569 13982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:23.741761 13988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:23.741762 13987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:23.741827 13990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:23.741902 13982 server_base.cc:1034] running on GCE node
I20260620 11:38:23.742286 13982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:23.742591 13982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:23.743762 13982 hybrid_clock.cc:648] HybridClock initialized: now 1781955503743749 us; error 29 us; skew 500 ppm
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:23.746577 13982 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:38:23.747030 13982 webserver.cc:460] Webserver started at http://127.5.122.130:45001/ using document root <none> and password file <none>
I20260620 11:38:23.747282 13982 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:23.747351 13982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:23.747502 13982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:23.748611 13982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "ed416b8f33e24a41ac865cefa05c3884"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "249d02082c200840ecf7cf6566fdd99b"
server_key_iv: "f800ac513dc8708e11728dea423b9c77"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.748989 13982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "ed416b8f33e24a41ac865cefa05c3884"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "249d02082c200840ecf7cf6566fdd99b"
server_key_iv: "f800ac513dc8708e11728dea423b9c77"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.750389 13982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:38:23.751137 13997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.751305 13982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:23.751408 13982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "ed416b8f33e24a41ac865cefa05c3884"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "249d02082c200840ecf7cf6566fdd99b"
server_key_iv: "f800ac513dc8708e11728dea423b9c77"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.751488 13982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:23.773851 13982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:23.776019 13982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:23.776196 13982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:23.776451 13982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:23.776770 13982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:23.776832 13982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.776875 13982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:23.776903 13982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.782795 13982 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:36093
I20260620 11:38:23.782832 14110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:36093 every 8 connection(s)
I20260620 11:38:23.783164 13982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:23.784828 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 13982
I20260620 11:38:23.784947 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:23.785136 5610 external_mini_cluster.cc:1428] Setting key 0eb72822060a226ac6dde54f4cd7f3b1
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
I20260620 11:38:23.793846 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:23.783882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:42461 (local address 127.5.122.190:38593)
0620 11:38:23.783949 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:23.783951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:23.784153 (+ 202us) server_negotiation.cc:407] Connection header received
0620 11:38:23.784697 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:23.784699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:23.784712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:23.784804 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:23.785944 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.786475 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.787961 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.788155 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.788223 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:23.790776 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:23.790778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:23.790778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:23.790785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:23.792366 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.792785 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.792786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.792787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.792856 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.793120 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.793121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.793122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.793238 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:23.793314 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:23.793645 (+ 331us) server_negotiation.cc:299] Negotiation successful
0620 11:38:23.793741 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 11:38:23.794065 14113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:23.783874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:42461)
0620 11:38:23.784026 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:23.784069 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:23.784596 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:23.784879 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:23.784882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:23.785564 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:23.785836 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:23.785843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.786572 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.786574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:23.787782 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:23.787786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.788247 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.788248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:23.788401 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:23.788946 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:23.788957 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:23.790657 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:23.792463 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:23.792471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:23.792474 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:23.792705 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:23.792925 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:23.792926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:23.792926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:23.793047 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:23.793311 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:23.793318 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:23.793495 (+ 177us) client_negotiation.cc:769] Sending connection context
0620 11:38:23.793625 (+ 130us) client_negotiation.cc:240] Negotiation successful
0620 11:38:23.793798 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260620 11:38:23.794620 14111 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38593
I20260620 11:38:23.794750 14111 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:23.794940 14111 heartbeater.cc:511] Master 127.5.122.190:38593 requested a full tablet report, sending...
I20260620 11:38:23.795362 13786 ts_manager.cc:194] Registered new tserver with Master: ed416b8f33e24a41ac865cefa05c3884 (127.5.122.130:36093)
I20260620 11:38:23.795702 13786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:42461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:38:23.812628 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38593
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:23.896065 14116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:23.896304 14116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:23.896330 14116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:23.897715 14116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:23.897763 14116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:23.897816 14116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:23.899544 14116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:23.899813 14116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:23.900063 14116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:23.905206 14121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:23.905229 14124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:23.905225 14122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:23.905452 14116 server_base.cc:1034] running on GCE node
I20260620 11:38:23.905682 14116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:23.905994 14116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:23.907176 14116 hybrid_clock.cc:648] HybridClock initialized: now 1781955503907160 us; error 26 us; skew 500 ppm
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:23.909801 14116 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:38:23.910422 14116 webserver.cc:460] Webserver started at http://127.5.122.131:38763/ using document root <none> and password file <none>
I20260620 11:38:23.910666 14116 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:23.910735 14116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:23.910876 14116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:23.911966 14116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "47db841d68c144c2b089bb714ec33f41"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "27d871228bd2067580044e6da637548e"
server_key_iv: "be455f8479eb71eb8f10a56db34259fb"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.912355 14116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "47db841d68c144c2b089bb714ec33f41"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "27d871228bd2067580044e6da637548e"
server_key_iv: "be455f8479eb71eb8f10a56db34259fb"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.913704 14116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260620 11:38:23.914567 14131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.914745 14116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:38:23.914846 14116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "47db841d68c144c2b089bb714ec33f41"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "27d871228bd2067580044e6da637548e"
server_key_iv: "be455f8479eb71eb8f10a56db34259fb"
server_key_version: "encryptionkey@0"
I20260620 11:38:23.914929 14116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:23.922950 14116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:23.925387 14116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:23.925580 14116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:23.925843 14116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:23.926249 14116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:23.926431 14116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.926525 14116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:23.926558 14116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:23.933467 14116 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:36367
I20260620 11:38:23.933516 14244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:36367 every 8 connection(s)
I20260620 11:38:23.933874 14116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:23.938144 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 14116
I20260620 11:38:23.938283 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:23.938490 5610 external_mini_cluster.cc:1428] Setting key 0df25b08a1f82c5faa2e64478c1d7ea4
Jun 20 11:38:23 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955503, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:23.943841 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:23.934677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:37471 (local address 127.5.122.190:38593)
0620 11:38:23.934765 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:23.934768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:23.934984 (+ 216us) server_negotiation.cc:407] Connection header received
0620 11:38:23.935484 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:23.935485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:23.935497 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:23.935572 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:23.936540 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.937046 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.938179 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.938543 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.938623 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:23.941053 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:23.941055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:23.941056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:23.941062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:23.942607 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.942989 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.942990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.942990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.943042 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:23.943253 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:23.943253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:23.943254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:23.943361 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:23.943439 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:23.943706 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 11:38:23.943773 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 11:38:23.944121 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:23.934653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:37471)
0620 11:38:23.934830 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:23.934882 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:23.935368 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:23.935632 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:23.935636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:23.936227 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:23.936442 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:23.936447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.937131 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.937131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:23.938084 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:23.938087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:23.938704 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:23.938706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:23.938855 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:23.939511 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:23.939521 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:23.940962 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:23.942680 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:23.942685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:23.942687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:23.942922 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:23.943099 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:23.943100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:23.943100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:23.943200 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:23.943443 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:23.943447 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:23.943566 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 11:38:23.943686 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 11:38:23.943828 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260620 11:38:23.944638 14245 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38593
I20260620 11:38:23.944741 14245 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:23.944909 14245 heartbeater.cc:511] Master 127.5.122.190:38593 requested a full tablet report, sending...
I20260620 11:38:23.945295 13786 ts_manager.cc:194] Registered new tserver with Master: 47db841d68c144c2b089bb714ec33f41 (127.5.122.131:36367)
I20260620 11:38:23.945690 13786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:37471
I20260620 11:38:23.950536 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab.
Entry for principal oryx/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab.
WARNING: no policy specified for oryx/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab.
Entry for principal oryx/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab.
WARNING: no policy specified for oryx/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab.
Entry for principal oryx/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab.
I20260620 11:38:24.024690 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 13848
I20260620 11:38:24.030083 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 13982
I20260620 11:38:24.035585 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 14116
I20260620 11:38:24.040179 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 13754
I20260620 11:38:24.045637 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38593
--webserver_interface=127.5.122.190
--webserver_port=45231
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:38593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:24.129791 14254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:24.130045 14254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:24.130070 14254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:24.131336 14254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:24.131389 14254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:24.131403 14254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:24.131417 14254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:24.131428 14254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:24.133010 14254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:38593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=45231
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:24.133275 14254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:24.133513 14254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:24.138553 14262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:24.138559 14259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:24.138742 14254 server_base.cc:1034] running on GCE node
W20260620 11:38:24.138573 14260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:24.139060 14254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:24.139379 14254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:24.140570 14254 hybrid_clock.cc:648] HybridClock initialized: now 1781955504140575 us; error 49 us; skew 500 ppm
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955504, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:24.143224 14254 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:24.143798 14254 webserver.cc:460] Webserver started at http://127.5.122.190:45231/ using document root <none> and password file <none>
I20260620 11:38:24.144034 14254 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:24.144117 14254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:24.145512 14254 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:24.146219 14269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.146395 14254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:24.146525 14254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "0663b82c2be24bb988313ef6725af425"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "92ca37b14185fec8398d4c15874ade9b"
server_key_iv: "d03aa8bd35ba03da4d8c808f0aa8dbd1"
server_key_version: "encryptionkey@0"
I20260620 11:38:24.146903 14254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:24.162846 14254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:24.164722 14254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:24.164891 14254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:24.169044 14254 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:38593
I20260620 11:38:24.169081 14321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:38593 every 8 connection(s)
I20260620 11:38:24.169486 14254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:24.170811 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 14254
I20260620 11:38:24.171154 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:44043
--local_ip_for_outbound_sockets=127.5.122.129
--tserver_master_addrs=127.5.122.190:38593
--webserver_port=34733
--webserver_interface=127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:38:24.171232 14322 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:38:24.172359 14322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Bootstrap starting.
I20260620 11:38:24.180742 14322 log.cc:824] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:24.182792 14322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:38:24.183115 14322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Bootstrap complete.
I20260620 11:38:24.185516 14322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:24.185757 14322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0663b82c2be24bb988313ef6725af425, State: Initialized, Role: FOLLOWER
I20260620 11:38:24.185930 14322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:24.186034 14322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:24.186100 14322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:24.186148 14322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 1 FOLLOWER]: Advancing to term 2
I20260620 11:38:24.187206 14322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:24.187317 14322 leader_election.cc:302] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0663b82c2be24bb988313ef6725af425; no voters:
I20260620 11:38:24.187515 14322 leader_election.cc:288] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 11:38:24.187552 14326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 11:38:24.187666 14326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [term 2 LEADER]: Becoming Leader. State: Replica: 0663b82c2be24bb988313ef6725af425, State: Running, Role: LEADER
I20260620 11:38:24.187812 14322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:24.187829 14326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } }
I20260620 11:38:24.188155 14327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0663b82c2be24bb988313ef6725af425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } } }
I20260620 11:38:24.188351 14327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:24.188150 14326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0663b82c2be24bb988313ef6725af425. Latest consensus state: current_term: 2 leader_uuid: "0663b82c2be24bb988313ef6725af425" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0663b82c2be24bb988313ef6725af425" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38593 } } }
I20260620 11:38:24.188735 14334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:24.188808 14326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:24.189299 14334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:24.189437 14334 catalog_manager.cc:1259] Loaded cluster ID: b2222abf34dc41c9b45ea124409d60cd
I20260620 11:38:24.189523 14334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:24.190842 14334 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:24.191035 14334 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0663b82c2be24bb988313ef6725af425: Loaded TSK: 0
I20260620 11:38:24.191248 14334 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 11:38:24.255388 14324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:24.255731 14324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:24.255779 14324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:24.257085 14324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:24.257131 14324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:24.257158 14324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:24.258631 14324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.129.keytab
--principal=oryx/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:44043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=34733
--tserver_master_addrs=127.5.122.190:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:24.258855 14324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:24.259099 14324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:24.264180 14348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:24.264199 14349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:24.264259 14351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:24.264623 14324 server_base.cc:1034] running on GCE node
I20260620 11:38:24.264859 14324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:24.265153 14324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:24.266335 14324 hybrid_clock.cc:648] HybridClock initialized: now 1781955504266314 us; error 32 us; skew 500 ppm
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955504, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:24.269416 14324 init.cc:377] Logged in from keytab as oryx/127.5.122.129@KRBTEST.COM (short username oryx)
I20260620 11:38:24.269919 14324 webserver.cc:460] Webserver started at http://127.5.122.129:34733/ using document root <none> and password file <none>
I20260620 11:38:24.270191 14324 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:24.270282 14324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:24.271797 14324 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:24.272514 14358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.272719 14324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:24.272828 14324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "463cb02dd09d400da3ae38433249801f"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "dcbb0675ba3eed4848612860a47acd1a"
server_key_iv: "7900eb248cb58aea10798dc665dc6ab8"
server_key_version: "encryptionkey@0"
I20260620 11:38:24.273204 14324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:24.281937 14324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:24.284499 14324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:24.284703 14324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:24.284983 14324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:24.285360 14324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:24.285415 14324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.285472 14324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:24.285516 14324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.292675 14324 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:44043
I20260620 11:38:24.292764 14471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:44043 every 8 connection(s)
I20260620 11:38:24.293139 14324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:24.296691 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 14324
I20260620 11:38:24.297067 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:36093
--local_ip_for_outbound_sockets=127.5.122.130
--tserver_master_addrs=127.5.122.190:38593
--webserver_port=45001
--webserver_interface=127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.303505 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.294027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:37835)
0620 11:38:24.294279 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.294321 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.294957 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.295433 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.295438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.296218 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.296533 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.296540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.298509 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.298511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.299345 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.299347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.300058 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.300059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.300198 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.300784 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.300793 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.303254 (+ 2461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":84,"threads_started":1}
W20260620 11:38:24.303777 14472 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:38593 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.122.190:38593: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
W20260620 11:38:24.304086 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.294102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:37835 (local address 127.5.122.190:38593)
0620 11:38:24.294289 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.294299 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.294486 (+ 187us) server_negotiation.cc:407] Connection header received
0620 11:38:24.295177 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.295183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.295229 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.295353 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.296709 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.298328 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.299509 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.299947 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.300079 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.303395 (+ 3316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:37835: BlockingRecv error: recv got EOF from 127.5.122.129:37835 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.308672 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.303985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:50279)
0620 11:38:24.304303 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.304316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.304364 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.304540 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.304541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.304678 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.304731 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.304733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.305402 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.305404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.305910 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.305911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.306294 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.306295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.306365 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.306827 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.306832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.308576 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":80,"threads_started":1}
W20260620 11:38:24.308730 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.304179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:50279 (local address 127.5.122.190:38593)
0620 11:38:24.304382 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.304384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.304403 (+ 19us) server_negotiation.cc:407] Connection header received
0620 11:38:24.304432 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.304433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.304446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.304531 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.304827 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.305314 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.306021 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.306211 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.306286 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.308660 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:50279: BlockingRecv error: recv got EOF from 127.5.122.129:50279 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":64,"threads_started":1}
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.313783 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.309219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:38997)
0620 11:38:24.309265 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.309277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.309321 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.309499 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.309500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.309626 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.309658 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.309660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.310370 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.310371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.310865 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.310866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.311302 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.311303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.311360 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.311824 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.311829 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.313701 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260620 11:38:24.313941 14472 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 11:38:24.314276 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.309263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:38997 (local address 127.5.122.190:38593)
0620 11:38:24.309378 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.309379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.309395 (+ 16us) server_negotiation.cc:407] Connection header received
0620 11:38:24.309413 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.309414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.309422 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.309481 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.309731 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.310279 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.310962 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.311216 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.311302 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.314224 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:38997: BlockingRecv error: recv got EOF from 127.5.122.129:38997 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260620 11:38:24.391184 14476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:24.391458 14476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:24.391486 14476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:24.392683 14476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:24.392724 14476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:24.392750 14476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:24.394218 14476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.130.keytab
--principal=oryx/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:36093
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=45001
--tserver_master_addrs=127.5.122.190:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:24.394436 14476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:24.394642 14476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:24.399880 14483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:24.399880 14486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:24.399888 14484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:24.400146 14476 server_base.cc:1034] running on GCE node
I20260620 11:38:24.400401 14476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:24.400674 14476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:24.401832 14476 hybrid_clock.cc:648] HybridClock initialized: now 1781955504401810 us; error 34 us; skew 500 ppm
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955504, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:24.404439 14476 init.cc:377] Logged in from keytab as oryx/127.5.122.130@KRBTEST.COM (short username oryx)
I20260620 11:38:24.404982 14476 webserver.cc:460] Webserver started at http://127.5.122.130:45001/ using document root <none> and password file <none>
I20260620 11:38:24.405251 14476 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:24.405344 14476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:24.406863 14476 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:24.407485 14493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.407670 14476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:24.407781 14476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "ed416b8f33e24a41ac865cefa05c3884"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "249d02082c200840ecf7cf6566fdd99b"
server_key_iv: "f800ac513dc8708e11728dea423b9c77"
server_key_version: "encryptionkey@0"
I20260620 11:38:24.408167 14476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:24.422868 14476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:24.424955 14476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:24.425139 14476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:24.425386 14476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:24.425710 14476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:24.425766 14476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.425813 14476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:24.425836 14476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.431918 14476 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:36093
I20260620 11:38:24.431963 14606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:36093 every 8 connection(s)
I20260620 11:38:24.432336 14476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:24.433116 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 14476
I20260620 11:38:24.433436 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:36367
--local_ip_for_outbound_sockets=127.5.122.131
--tserver_master_addrs=127.5.122.190:38593
--webserver_port=38763
--webserver_interface=127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:35449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
W20260620 11:38:24.440716 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.433428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:38279 (local address 127.5.122.190:38593)
0620 11:38:24.434203 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.434205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.434215 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:24.434247 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.434248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.434259 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.434331 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.435365 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.435874 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.436993 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.437207 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.437297 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.440657 (+ 3360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:38279: BlockingRecv error: recv got EOF from 127.5.122.130:38279 (error 108)
Metrics: {"server-negotiator.queue_time_us":748}
I20260620 11:38:24.440786 14609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.433153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:38279)
0620 11:38:24.433336 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.433390 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.434011 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.434427 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.434431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.435018 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.435254 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.435259 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.435992 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.435993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.436902 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.436904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.437310 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.437311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.437436 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.438112 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.438121 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.440569 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
W20260620 11:38:24.441138 14607 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:38593 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.122.190:38593: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.446077 14609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.441384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:35295)
0620 11:38:24.441438 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.441449 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.441519 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.441668 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.441669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.441812 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.441861 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.441862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.442564 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.442565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.443090 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.443092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.443518 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.443519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.443584 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.444088 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.444093 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.446001 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 11:38:24.446192 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.441462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:35295 (local address 127.5.122.190:38593)
0620 11:38:24.441524 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.441526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.441546 (+ 20us) server_negotiation.cc:407] Connection header received
0620 11:38:24.441574 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.441575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.441586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.441653 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.441948 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.442482 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.443179 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.443433 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.443518 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.446127 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:35295: BlockingRecv error: recv got EOF from 127.5.122.130:35295 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.450999 14609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.446459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:34665)
0620 11:38:24.446512 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.446526 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.446573 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.446795 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.446796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.446916 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.446948 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.446950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.447583 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.447584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.448086 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.448087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.448450 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.448451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.448505 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.449003 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.449007 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.450918 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 11:38:24.451086 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.446541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:34665 (local address 127.5.122.190:38593)
0620 11:38:24.446583 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.446585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.446603 (+ 18us) server_negotiation.cc:407] Connection header received
0620 11:38:24.446685 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.446686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.446697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.446765 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.447021 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.447499 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.448171 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.448365 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.448451 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.451008 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:34665: BlockingRecv error: recv got EOF from 127.5.122.130:34665 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 11:38:24.451346 14607 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 11:38:24.519927 14610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:24.520187 14610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:24.520224 14610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:24.521445 14610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:24.521521 14610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:24.521569 14610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:24.523228 14610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/krb5kdc/oryx_127.5.122.131.keytab
--principal=oryx/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:36367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=38763
--tserver_master_addrs=127.5.122.190:38593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:24.523504 14610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:24.523752 14610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:24.528851 14615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:24.528856 14616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:24.528872 14610 server_base.cc:1034] running on GCE node
W20260620 11:38:24.528890 14618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:24.529306 14610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:24.529561 14610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:24.530700 14610 hybrid_clock.cc:648] HybridClock initialized: now 1781955504530679 us; error 36 us; skew 500 ppm
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955504, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:24.533248 14610 init.cc:377] Logged in from keytab as oryx/127.5.122.131@KRBTEST.COM (short username oryx)
I20260620 11:38:24.533664 14610 webserver.cc:460] Webserver started at http://127.5.122.131:38763/ using document root <none> and password file <none>
I20260620 11:38:24.533921 14610 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:24.534024 14610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:24.535420 14610 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:24.536089 14625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.536280 14610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:24.536381 14610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "47db841d68c144c2b089bb714ec33f41"
format_stamp: "Formatted at 2026-06-20 11:38:23 on dist-test-slave-szdr"
server_key: "27d871228bd2067580044e6da637548e"
server_key_iv: "be455f8479eb71eb8f10a56db34259fb"
server_key_version: "encryptionkey@0"
I20260620 11:38:24.536767 14610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:24.556120 14610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:24.558301 14610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:24.558482 14610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:24.558802 14610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:24.559151 14610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:24.559211 14610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.559253 14610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:24.559281 14610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:24.565011 14610 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:36367
I20260620 11:38:24.565096 14738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:36367 every 8 connection(s)
I20260620 11:38:24.565454 14610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:24.568984 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 14610
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.573841 14741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.566330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:48345)
0620 11:38:24.566505 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.566550 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.567041 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.567270 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.567273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.567859 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.568064 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.568069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.568730 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.568731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.570018 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.570021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.570397 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.570398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.570555 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.571135 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.571143 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.573684 (+ 2541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
W20260620 11:38:24.573876 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.566344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:48345 (local address 127.5.122.190:38593)
0620 11:38:24.566413 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.566415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.566662 (+ 247us) server_negotiation.cc:407] Connection header received
0620 11:38:24.567126 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.567127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.567137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.567206 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.568156 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.568641 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.570100 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.570310 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.570363 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.573794 (+ 3431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:48345: BlockingRecv error: recv got EOF from 127.5.122.131:48345 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260620 11:38:24.574414 14739 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:38593 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.122.190:38593: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
I20260620 11:38:24.577914 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.569551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.5.122.190:38593)
0620 11:38:24.569626 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.569628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.569637 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:24.569709 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.569710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.569719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.569777 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.570574 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.571223 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.572217 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.572503 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.572557 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.573931 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:24.573937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:24.573937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:24.573950 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:24.576073 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:24.576468 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:24.576471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:24.576472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:24.576601 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:24.576864 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:24.576866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:24.576866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:24.577201 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:24.577382 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:24.577687 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 11:38:24.577803 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.579773 14741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.574652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:39087)
0620 11:38:24.574711 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.574723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.574769 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.574902 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.574903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.575056 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.575110 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.575111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.576019 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.576020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.576898 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.576900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.577334 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.577335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.577382 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.577806 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.577810 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.579726 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 11:38:24.579916 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.574734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:39087 (local address 127.5.122.190:38593)
0620 11:38:24.574779 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.574781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.574790 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:24.574823 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.574824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.574834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.574887 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.575204 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.575944 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.576980 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.577265 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.577330 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.579841 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:39087: BlockingRecv error: recv got EOF from 127.5.122.131:39087 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:24 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:24.583843 14741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:24.580044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:46031)
0620 11:38:24.580073 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:24.580084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:24.580121 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:24.580305 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:24.580306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:24.580405 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:24.580431 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.580432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.581024 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.581025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.581494 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:24.581495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.581800 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.581800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:24.581834 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.582268 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:24.582272 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:24.583796 (+ 1524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":14}
W20260620 11:38:24.583967 14475 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:24.580143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:46031 (local address 127.5.122.190:38593)
0620 11:38:24.580197 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:24.580199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:24.580207 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:24.580226 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:24.580226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:24.580242 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:24.580289 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:24.580511 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.580954 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.581564 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:24.581738 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:24.581787 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:24.583893 (+ 2106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:46031: BlockingRecv error: recv got EOF from 127.5.122.131:46031 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260620 11:38:24.583979 14739 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 11:38:25 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:25 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:25.319188 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:25.314377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:47197)
0620 11:38:25.314534 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:25.314545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:25.314610 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:25.314755 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:25.314755 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:25.314929 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:25.314979 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:25.314980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.315653 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.315654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:25.316232 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:25.316234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.316625 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.316626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:25.316674 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:25.317129 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:25.317135 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:25.319116 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":95,"threads_started":1}
W20260620 11:38:25.319367 14744 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:25.314500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:47197 (local address 127.5.122.190:38593)
0620 11:38:25.314631 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:25.314633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:25.314642 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:25.314667 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:25.314668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:25.314677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:25.314735 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:25.315052 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.315553 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.316326 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.316551 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.316606 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:25.319274 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:47197: BlockingRecv error: recv got EOF from 127.5.122.129:47197 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
Jun 20 11:38:25 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:25 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:25.456692 14745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:25.451818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:47083)
0620 11:38:25.451983 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:25.451993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:25.452049 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:25.452274 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:25.452275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:25.452466 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:25.452517 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:25.452519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.453191 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.453192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:25.453756 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:25.453758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.454248 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.454249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:25.454300 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:25.454747 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:25.454752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:25.456620 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":70,"threads_started":1}
W20260620 11:38:25.456763 14744 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:25.451875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:47083 (local address 127.5.122.190:38593)
0620 11:38:25.451932 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:25.451935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:25.452061 (+ 126us) server_negotiation.cc:407] Connection header received
0620 11:38:25.452150 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:25.452150 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:25.452181 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:25.452245 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:25.452583 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.453094 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.453840 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.454171 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.454231 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:25.456714 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:47083: BlockingRecv error: recv got EOF from 127.5.122.130:47083 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 11:38:25 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:25 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:25.589047 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:25.584369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:51039)
0620 11:38:25.584510 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:25.584524 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:25.584585 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:25.584735 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:25.584736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:25.584900 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:25.584949 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:25.584951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.585586 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.585587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:25.586178 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:25.586180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.586492 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.586493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:25.586530 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:25.586999 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:25.587003 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:25.588934 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
W20260620 11:38:25.589083 14744 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:25.584447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:51039 (local address 127.5.122.190:38593)
0620 11:38:25.584517 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:25.584519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:25.584570 (+ 51us) server_negotiation.cc:407] Connection header received
0620 11:38:25.584652 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:25.584653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:25.584663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:25.584713 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:25.585015 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.585515 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.586247 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:25.586426 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:25.586479 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:25.589017 (+ 2538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:51039: BlockingRecv error: recv got EOF from 127.5.122.131:51039 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 20 11:38:26 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:26 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:26.324374 14747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:26.319679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:36061)
0620 11:38:26.319838 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:26.319849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:26.319899 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:26.320064 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:26.320064 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:26.320227 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:26.320291 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:26.320292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.320939 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.320940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:26.321466 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:26.321467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.321808 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.321809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:26.321855 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:26.322342 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:26.322347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:26.324271 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
W20260620 11:38:26.324436 14748 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:26.319750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:36061 (local address 127.5.122.190:38593)
0620 11:38:26.319886 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:26.319888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:26.319897 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:26.319944 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:26.319944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:26.319955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:26.320035 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:26.320362 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.320869 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.321540 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.321732 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.321783 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:26.324365 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:36061: BlockingRecv error: recv got EOF from 127.5.122.129:36061 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
Jun 20 11:38:26 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:26 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:26.461840 14749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:26.457266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:50647)
0620 11:38:26.457428 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:26.457438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:26.457514 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:26.457645 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:26.457646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:26.457809 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:26.457859 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:26.457861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.458525 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.458526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:26.459042 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:26.459044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.459358 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.459359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:26.459399 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:26.459840 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:26.459844 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:26.461780 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
W20260620 11:38:26.461946 14748 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:26.457372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:50647 (local address 127.5.122.190:38593)
0620 11:38:26.457425 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:26.457428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:26.457521 (+ 93us) server_negotiation.cc:407] Connection header received
0620 11:38:26.457566 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:26.457567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:26.457578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:26.457618 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:26.457919 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.458439 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.459112 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.459293 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.459343 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:26.461874 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:50647: BlockingRecv error: recv got EOF from 127.5.122.130:50647 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Jun 20 11:38:26 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:26 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:26.594508 14750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:26.589611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:43641)
0620 11:38:26.589771 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:26.589782 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:26.589841 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:26.590022 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:26.590023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:26.590203 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:26.590257 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:26.590259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.591106 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.591107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:26.591690 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:26.591693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.592044 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.592045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:26.592088 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:26.592532 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:26.592536 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:26.594440 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
W20260620 11:38:26.594647 14748 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:26.589775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:43641 (local address 127.5.122.190:38593)
0620 11:38:26.589857 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:26.589859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:26.589869 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:26.589896 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:26.589896 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:26.589909 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:26.589994 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:26.590329 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.591040 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.591774 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:26.591958 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:26.592022 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:26.594538 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:43641: BlockingRecv error: recv got EOF from 127.5.122.131:43641 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 11:38:27 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:27 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:27.330289 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:27.325031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:36249)
0620 11:38:27.325201 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:27.325214 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:27.325300 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:27.325469 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:27.325470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:27.325636 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:27.325689 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:27.325690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.326521 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.326522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:27.327351 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:27.327354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.327679 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.327680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:27.327720 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:27.328220 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:27.328225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:27.330205 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
W20260620 11:38:27.330400 14752 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:27.325152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:36249 (local address 127.5.122.190:38593)
0620 11:38:27.325312 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:27.325314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:27.325324 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:27.325374 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:27.325375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:27.325385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:27.325444 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:27.325760 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.326402 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.327419 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.327612 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.327668 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:27.330318 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:36249: BlockingRecv error: recv got EOF from 127.5.122.129:36249 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
Jun 20 11:38:27 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:27 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:27.467334 14753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:27.462331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:50501)
0620 11:38:27.462554 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:27.462567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:27.462672 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:27.462827 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:27.462828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:27.463021 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:27.463067 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:27.463069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.463767 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.463769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:27.464404 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:27.464407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.464805 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.464806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:27.464870 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:27.465329 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:27.465334 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:27.467270 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":101,"threads_started":1}
W20260620 11:38:27.467521 14752 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:27.462460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:50501 (local address 127.5.122.190:38593)
0620 11:38:27.462555 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:27.462557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:27.462650 (+ 93us) server_negotiation.cc:407] Connection header received
0620 11:38:27.462742 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:27.462743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:27.462756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:27.462809 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:27.463151 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.463674 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.464499 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.464707 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.464798 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:27.467436 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:50501: BlockingRecv error: recv got EOF from 127.5.122.130:50501 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 20 11:38:27 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:27 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:27.599840 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:27.595011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:59177)
0620 11:38:27.595186 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:27.595197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:27.595261 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:27.595430 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:27.595432 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:27.595606 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:27.595655 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:27.595657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.596338 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.596339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:27.596899 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:27.596901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.597244 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.597245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:27.597284 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:27.597739 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:27.597744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:27.599733 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
W20260620 11:38:27.599897 14752 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:27.595097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:59177 (local address 127.5.122.190:38593)
0620 11:38:27.595162 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:27.595164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:27.595262 (+ 98us) server_negotiation.cc:407] Connection header received
0620 11:38:27.595307 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:27.595308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:27.595322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:27.595376 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:27.595734 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.596256 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.596989 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:27.597181 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:27.597231 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:27.599833 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:59177: BlockingRecv error: recv got EOF from 127.5.122.131:59177 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 20 11:38:28 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:28 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:28.336177 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:28.330977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:42899)
0620 11:38:28.331104 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:28.331119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:28.331179 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:28.331430 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:28.331431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:28.331595 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:28.331645 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:28.331647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.332444 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.332446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:28.333007 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:28.333009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.333427 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.333428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:28.333480 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:28.334189 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:28.334194 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:28.336075 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260620 11:38:28.336280 14756 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:28.331091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:42899 (local address 127.5.122.190:38593)
0620 11:38:28.331243 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:28.331245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:28.331255 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:28.331281 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:28.331281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:28.331327 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:28.331418 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:28.331765 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.332332 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.333105 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.333338 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.333397 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:28.336167 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:42899: BlockingRecv error: recv got EOF from 127.5.122.129:42899 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":238,"threads_started":1}
Jun 20 11:38:28 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:28 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:28.479234 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:28.467928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:37855)
0620 11:38:28.468119 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:28.468133 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:28.468229 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:28.468419 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:28.468420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:28.468621 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:28.468675 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:28.468677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.469385 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.469387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:28.469944 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:28.469946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.470376 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.470377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:28.470442 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:28.476775 (+ 6333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:28.476783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:28.479163 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
W20260620 11:38:28.479413 14756 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:28.468036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:37855 (local address 127.5.122.190:38593)
0620 11:38:28.468120 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:28.468123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:28.468207 (+ 84us) server_negotiation.cc:407] Connection header received
0620 11:38:28.468317 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:28.468318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:28.468333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:28.468409 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:28.468755 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.469287 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.470058 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.470266 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.470375 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:28.479339 (+ 8964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:37855: BlockingRecv error: recv got EOF from 127.5.122.130:37855 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 20 11:38:28 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:28 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:28.605160 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:28.600389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:40939)
0620 11:38:28.600547 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:28.600558 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:28.600624 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:28.600760 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:28.600761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:28.600916 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:28.600966 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:28.600967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.601664 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.601665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:28.602254 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:28.602257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.602616 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.602617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:28.602663 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:28.603238 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:28.603244 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:28.605098 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
W20260620 11:38:28.605290 14756 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:28.600450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:40939 (local address 127.5.122.190:38593)
0620 11:38:28.600519 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:28.600521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:28.600616 (+ 95us) server_negotiation.cc:407] Connection header received
0620 11:38:28.600675 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:28.600676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:28.600692 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:28.600758 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:28.601022 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.601553 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.602353 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:28.602546 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:28.602601 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:28.605202 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:40939: BlockingRecv error: recv got EOF from 127.5.122.131:40939 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 20 11:38:29 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:29 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:29.342286 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:29.336837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:36847)
0620 11:38:29.337029 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:29.337040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:29.337100 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:29.337259 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:29.337260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:29.337432 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:29.337479 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:29.337481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.338157 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.338157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:29.338673 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:29.338675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.339053 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.339054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:29.339092 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:29.340184 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:29.340188 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:29.342224 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":126,"threads_started":1}
W20260620 11:38:29.342458 14760 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:29.336945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:36847 (local address 127.5.122.190:38593)
0620 11:38:29.337083 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:29.337085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:29.337095 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:29.337173 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:29.337174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:29.337184 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:29.337246 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:29.337543 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.338092 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.338765 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.338975 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.339053 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:29.342357 (+ 3304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:36847: BlockingRecv error: recv got EOF from 127.5.122.129:36847 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":52,"threads_started":1}
Jun 20 11:38:29 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:29 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:29.484372 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:29.479772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:39829)
0620 11:38:29.479952 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:29.479963 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:29.480015 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:29.480142 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:29.480143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:29.480319 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:29.480374 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:29.480376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.481059 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.481060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:29.481610 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:29.481612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.481942 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.481943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:29.482009 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:29.482501 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:29.482505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:29.484306 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
W20260620 11:38:29.484520 14760 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:29.479919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:39829 (local address 127.5.122.190:38593)
0620 11:38:29.479999 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:29.480000 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:29.480010 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:29.480065 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:29.480066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:29.480076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:29.480123 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:29.480438 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.480985 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.481694 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.481880 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.481931 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:29.484431 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:39829: BlockingRecv error: recv got EOF from 127.5.122.130:39829 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 11:38:29 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:29 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:29.611024 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:29.605822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:49985)
0620 11:38:29.606030 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:29.606050 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:29.606125 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:29.606309 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:29.606310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:29.606538 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:29.606610 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:29.606613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.607428 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.607429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:29.607961 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:29.607962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.608312 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.608313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:29.608354 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:29.608905 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:29.608911 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:29.610961 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
W20260620 11:38:29.611162 14760 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:29.605951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:49985 (local address 127.5.122.190:38593)
0620 11:38:29.606033 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:29.606036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:29.606131 (+ 95us) server_negotiation.cc:407] Connection header received
0620 11:38:29.606168 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:29.606169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:29.606183 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:29.606289 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:29.606697 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.607355 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.608045 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:29.608245 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:29.608299 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:29.611065 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:49985: BlockingRecv error: recv got EOF from 127.5.122.131:49985 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 20 11:38:30 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:30 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:30.347774 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:30.342843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:38681)
0620 11:38:30.343017 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:30.343028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:30.343091 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:30.343249 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:30.343250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:30.343427 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:30.343479 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:30.343481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.344233 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.344235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:30.344945 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:30.344949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.345306 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.345306 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:30.345348 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:30.345836 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:30.345841 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:30.347710 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260620 11:38:30.347939 14764 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:30.342953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:38681 (local address 127.5.122.190:38593)
0620 11:38:30.343094 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:30.343095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:30.343137 (+ 42us) server_negotiation.cc:407] Connection header received
0620 11:38:30.343161 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:30.343162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:30.343173 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:30.343224 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:30.343547 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.344170 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.345033 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.345236 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.345293 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:30.347823 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:38681: BlockingRecv error: recv got EOF from 127.5.122.129:38681 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
Jun 20 11:38:30 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:30 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:30.490476 14765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:30.484978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:33357)
0620 11:38:30.485159 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:30.485175 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:30.485242 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:30.485397 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:30.485398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:30.485597 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:30.485652 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:30.485654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.486445 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.486446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:30.487122 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:30.487127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.487638 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.487639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:30.487708 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:30.488365 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:30.488370 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:30.490412 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
W20260620 11:38:30.490633 14764 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:30.485144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:33357 (local address 127.5.122.190:38593)
0620 11:38:30.485226 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:30.485228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:30.485240 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:38:30.485312 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:30.485313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:30.485325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:30.485372 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:30.485723 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.486362 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.487230 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.487564 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.487636 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:30.490537 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:33357: BlockingRecv error: recv got EOF from 127.5.122.130:33357 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 11:38:30 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:30 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:30.616073 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:30.611517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:34165)
0620 11:38:30.611742 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:30.611754 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:30.611817 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:30.611941 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:30.611941 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:30.612154 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:30.612202 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:30.612204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.612828 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.612828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:30.613404 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:30.613406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.613713 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.613713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:30.613753 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:30.614252 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:30.614256 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:30.616000 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":115,"threads_started":1}
W20260620 11:38:30.616154 14764 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:30.611610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:34165 (local address 127.5.122.190:38593)
0620 11:38:30.611720 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:30.611723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:30.611816 (+ 93us) server_negotiation.cc:407] Connection header received
0620 11:38:30.611861 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:30.611862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:30.611874 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:30.611923 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:30.612266 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.612761 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.613470 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:30.613651 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:30.613698 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:30.616105 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:34165: BlockingRecv error: recv got EOF from 127.5.122.131:34165 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 20 11:38:31 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:31 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:31.353658 14767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:31.348352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:52771)
0620 11:38:31.348528 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:31.348542 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:31.348616 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:31.348871 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:31.348872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:31.349139 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:31.349227 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:31.349247 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.350074 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.350075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:31.350681 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:31.350683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.351105 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.351106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:31.351161 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:31.351633 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:31.351638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:31.353544 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
W20260620 11:38:31.353706 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:31.348442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:52771 (local address 127.5.122.190:38593)
0620 11:38:31.348632 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:31.348635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:31.348667 (+ 32us) server_negotiation.cc:407] Connection header received
0620 11:38:31.348696 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:31.348697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:31.348711 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:31.348773 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:31.349333 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.349923 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.350787 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.350997 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.351107 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:31.353629 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:52771: BlockingRecv error: recv got EOF from 127.5.122.129:52771 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":73,"threads_started":1}
Jun 20 11:38:31 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:31 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:31.495930 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:31.490972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:50475)
0620 11:38:31.491168 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:31.491179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:31.491270 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:31.491457 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:31.491458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:31.491649 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:31.491699 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:31.491701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.492392 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.492393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:31.492947 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:31.492950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.493335 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.493336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:31.493396 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:31.493913 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:31.493918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:31.495832 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
W20260620 11:38:31.495996 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:31.491047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:50475 (local address 127.5.122.190:38593)
0620 11:38:31.491133 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:31.491135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:31.491253 (+ 118us) server_negotiation.cc:407] Connection header received
0620 11:38:31.491329 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:31.491330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:31.491344 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:31.491419 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:31.491785 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.492304 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.493044 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.493247 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.493323 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:31.495932 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:50475: BlockingRecv error: recv got EOF from 127.5.122.130:50475 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 20 11:38:31 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:31 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:31.621814 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:31.616653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:37957)
0620 11:38:31.616798 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:31.616818 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:31.616906 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:31.617115 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:31.617116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:31.617288 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:31.617339 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:31.617341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.618111 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.618111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:31.618661 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:31.618663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.619077 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.619078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:31.619130 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:31.619612 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:31.619620 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:31.621712 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
W20260620 11:38:31.621863 14768 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:31.616757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:37957 (local address 127.5.122.190:38593)
0620 11:38:31.616845 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:31.616847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:31.616888 (+ 41us) server_negotiation.cc:407] Connection header received
0620 11:38:31.617011 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:31.617012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:31.617029 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:31.617090 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:31.617438 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.618023 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.618743 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:31.618979 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:31.619037 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:31.621795 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:37957: BlockingRecv error: recv got EOF from 127.5.122.131:37957 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 11:38:32 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:32 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:32.359267 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:32.354327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:49225)
0620 11:38:32.354476 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:32.354494 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:32.354547 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:32.354781 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:32.354782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:32.354950 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:32.355003 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:32.355005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.355691 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.355692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:32.356232 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:32.356234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.356638 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.356639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:32.356753 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:32.357308 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:32.357317 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:32.359203 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
W20260620 11:38:32.359436 14772 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:32.354464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:49225 (local address 127.5.122.190:38593)
0620 11:38:32.354622 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:32.354624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:32.354631 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:38:32.354653 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:32.354654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:32.354664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:32.354730 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:32.355076 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.355619 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.356304 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.356517 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.356622 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:32.359347 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:49225: BlockingRecv error: recv got EOF from 127.5.122.129:49225 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
Jun 20 11:38:32 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:32 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:32.501814 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:32.496577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:36855)
0620 11:38:32.496747 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:32.496792 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:32.496866 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:32.497002 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:32.497002 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:32.497170 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:32.497221 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:32.497222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.498446 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.498447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:32.498981 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:32.498983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.499350 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.499351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:32.499390 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:32.499913 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:32.499924 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:32.501738 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
W20260620 11:38:32.501961 14772 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:32.496708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:36855 (local address 127.5.122.190:38593)
0620 11:38:32.496766 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:32.496768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:32.496867 (+ 99us) server_negotiation.cc:407] Connection header received
0620 11:38:32.496917 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:32.496918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:32.496929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:32.496982 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:32.497380 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.498365 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.499078 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.499274 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.499341 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:32.501874 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:36855: BlockingRecv error: recv got EOF from 127.5.122.130:36855 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 20 11:38:32 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:32 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:32.626914 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:32.622380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:47943)
0620 11:38:32.622550 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:32.622561 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:32.622628 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:32.622794 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:32.622795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:32.622975 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:32.623021 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:32.623022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.623655 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.623655 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:32.624178 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:32.624179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.624502 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.624503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:32.624542 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:32.624977 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:32.624981 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:32.626854 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
W20260620 11:38:32.627034 14772 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:32.622460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:47943 (local address 127.5.122.190:38593)
0620 11:38:32.622535 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:32.622537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:32.622629 (+ 92us) server_negotiation.cc:407] Connection header received
0620 11:38:32.622702 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:32.622703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:32.622732 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:32.622774 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:32.623083 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.623584 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.624251 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:32.624438 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:32.624489 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:32.626939 (+ 2450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:47943: BlockingRecv error: recv got EOF from 127.5.122.131:47943 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 11:38:33 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:33 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:33.364923 14775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:33.359841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:46127)
0620 11:38:33.359982 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:33.359992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:33.360040 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:33.360198 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:33.360198 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:33.360375 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:33.360428 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:33.360429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.361262 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.361262 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:33.361801 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:33.361803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.362296 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.362298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:33.362349 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:33.362930 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:33.362936 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:33.364852 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
W20260620 11:38:33.365135 14776 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:33.359882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:46127 (local address 127.5.122.190:38593)
0620 11:38:33.360037 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:33.360039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:33.360048 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:33.360119 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:33.360120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:33.360130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:33.360192 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:33.360536 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.361168 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.361887 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.362219 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.362285 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:33.365032 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:46127: BlockingRecv error: recv got EOF from 127.5.122.129:46127 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
Jun 20 11:38:33 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:33 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:33.510571 14777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:33.502288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:35453)
0620 11:38:33.502435 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:33.502445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:33.502500 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:33.502672 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:33.502673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:33.502844 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:33.502897 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:33.502898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.503576 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.503577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:33.504105 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:33.504107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.504440 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.504441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:33.504483 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:33.508380 (+ 3897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:33.508394 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:33.510494 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
W20260620 11:38:33.510674 14776 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:33.502381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:35453 (local address 127.5.122.190:38593)
0620 11:38:33.502431 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:33.502433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:33.502489 (+ 56us) server_negotiation.cc:407] Connection header received
0620 11:38:33.502577 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:33.502578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:33.502590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:33.502635 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:33.502975 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.503483 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.504183 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.504370 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.504420 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:33.510602 (+ 6182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:35453: BlockingRecv error: recv got EOF from 127.5.122.130:35453 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 20 11:38:33 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:33 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:33.632296 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:33.627529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:52039)
0620 11:38:33.627710 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:33.627722 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:33.627790 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:33.627951 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:33.627952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:33.628116 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:33.628190 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:33.628192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.628854 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.628855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:33.629410 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:33.629412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.629738 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.629739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:33.629782 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:33.630325 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:33.630331 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:33.632220 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":132,"threads_started":1}
W20260620 11:38:33.632397 14776 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:33.627617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:52039 (local address 127.5.122.190:38593)
0620 11:38:33.627814 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:33.627816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:33.627826 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:33.627852 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:33.627853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:33.627866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:33.627932 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:33.628265 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.628767 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.629489 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:33.629673 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:33.629724 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:33.632324 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:52039: BlockingRecv error: recv got EOF from 127.5.122.131:52039 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
Jun 20 11:38:34 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:34 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:34.370602 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:34.365514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:47557)
0620 11:38:34.365729 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:34.365745 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:34.365839 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:34.366019 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:34.366020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:34.366187 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:34.366235 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:34.366237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.366926 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.366927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:34.367463 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:34.367465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.367819 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.367820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:34.367862 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:34.368341 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:34.368346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:34.370502 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
W20260620 11:38:34.370668 14780 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:34.365662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:47557 (local address 127.5.122.190:38593)
0620 11:38:34.365804 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:34.365806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:34.365816 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:34.365916 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:34.365917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:34.365928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:34.366008 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:34.366307 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.366812 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.367543 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.367746 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.367801 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:34.370592 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:47557: BlockingRecv error: recv got EOF from 127.5.122.129:47557 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
Jun 20 11:38:34 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:34 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:34.516233 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:34.511216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:34915)
0620 11:38:34.511385 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:34.511396 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:34.511461 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:34.511606 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:34.511607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:34.511826 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:34.511875 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:34.511877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.512617 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.512636 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:34.513358 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:34.513362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.513703 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.513704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:34.513747 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:34.514314 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:34.514326 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:34.516162 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
W20260620 11:38:34.516364 14780 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:34.511317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:34915 (local address 127.5.122.190:38593)
0620 11:38:34.511400 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:34.511403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:34.511465 (+ 62us) server_negotiation.cc:407] Connection header received
0620 11:38:34.511512 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:34.511513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:34.511524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:34.511588 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:34.511946 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.512518 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.513433 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.513632 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.513688 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:34.516256 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:34915: BlockingRecv error: recv got EOF from 127.5.122.130:34915 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 20 11:38:34 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:34 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:34.637563 14782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:34.632930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:54725)
0620 11:38:34.633138 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:34.633158 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:34.633216 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:34.633338 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:34.633339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:34.633507 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:34.633554 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:34.633555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.634293 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.634294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:34.634826 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:34.634828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.635207 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.635208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:34.635250 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:34.635703 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:34.635707 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:34.637500 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":92,"threads_started":1}
W20260620 11:38:34.637677 14780 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:34.633051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:54725 (local address 127.5.122.190:38593)
0620 11:38:34.633101 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:34.633103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:34.633211 (+ 108us) server_negotiation.cc:407] Connection header received
0620 11:38:34.633261 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:34.633262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:34.633274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:34.633322 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:34.633622 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.634219 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.634927 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:34.635133 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:34.635195 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:34.637604 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:54725: BlockingRecv error: recv got EOF from 127.5.122.131:54725 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 20 11:38:35 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:35 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:35.376122 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:35.371347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:57909)
0620 11:38:35.371493 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:35.371508 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:35.371568 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:35.371775 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:35.371776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:35.371949 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:35.371995 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:35.371997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.372672 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.372674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:35.373238 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:35.373240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.373631 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.373631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:35.373673 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:35.374159 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:35.374163 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:35.376045 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
W20260620 11:38:35.376322 14784 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:35.371422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:57909 (local address 127.5.122.190:38593)
0620 11:38:35.371555 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:35.371558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:35.371576 (+ 18us) server_negotiation.cc:407] Connection header received
0620 11:38:35.371676 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:35.371677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:35.371687 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:35.371753 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:35.372061 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.372578 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.373327 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.373525 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.373583 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:35.376231 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:57909: BlockingRecv error: recv got EOF from 127.5.122.129:57909 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
Jun 20 11:38:35 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:35 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:35.521744 14785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:35.516830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:44733)
0620 11:38:35.517034 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:35.517055 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:35.517128 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:35.517277 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:35.517278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:35.517450 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:35.517497 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:35.517498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.518216 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.518217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:35.518759 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:35.518761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.519109 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.519110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:35.519149 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:35.519731 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:35.519736 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:35.521640 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
W20260620 11:38:35.521879 14784 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:35.516979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:44733 (local address 127.5.122.190:38593)
0620 11:38:35.517057 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:35.517058 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:35.517124 (+ 66us) server_negotiation.cc:407] Connection header received
0620 11:38:35.517179 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:35.517180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:35.517191 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:35.517247 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:35.517561 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.518125 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.518847 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.519039 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.519093 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:35.521802 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:44733: BlockingRecv error: recv got EOF from 127.5.122.130:44733 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 20 11:38:35 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:35 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:35.643390 14786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:35.638196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:35431)
0620 11:38:35.638406 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:35.638420 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:35.638506 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:35.638738 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:35.638739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:35.638949 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:35.639005 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:35.639007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.639696 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.639698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:35.640349 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:35.640352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.640729 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.640730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:35.640793 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:35.641275 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:35.641280 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:35.643317 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":101,"threads_started":1}
W20260620 11:38:35.643471 14784 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:35.638338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:35431 (local address 127.5.122.190:38593)
0620 11:38:35.638418 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:35.638421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:35.638482 (+ 61us) server_negotiation.cc:407] Connection header received
0620 11:38:35.638575 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:35.638576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:35.638599 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:35.638727 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:35.639089 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.639607 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.640441 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:35.640653 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:35.640731 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:35.643392 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:35431: BlockingRecv error: recv got EOF from 127.5.122.131:35431 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 11:38:36 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:36 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:36.381779 14787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:36.376666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:58969)
0620 11:38:36.376794 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:36.376810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:36.376904 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:36.377193 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:36.377194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:36.377376 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:36.377427 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:36.377429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.378146 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.378147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:36.378686 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:36.378688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.379253 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.379254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:36.379329 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:36.379886 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:36.379890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:36.381716 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
W20260620 11:38:36.381929 14788 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:36.376878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:58969 (local address 127.5.122.190:38593)
0620 11:38:36.377044 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:36.377046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:36.377056 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:36.377083 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:36.377084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:36.377095 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:36.377169 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:36.377506 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.378064 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.378794 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.379134 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.379199 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:36.381831 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:58969: BlockingRecv error: recv got EOF from 127.5.122.129:58969 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
Jun 20 11:38:36 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:36 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:36.527083 14789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:36.522230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:43741)
0620 11:38:36.522430 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:36.522443 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:36.522540 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:36.522671 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:36.522672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:36.522872 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:36.522918 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:36.522920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.523614 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.523615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:36.524215 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:36.524218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.524618 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.524619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:36.524682 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:36.525165 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:36.525170 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:36.526950 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":90,"threads_started":1}
W20260620 11:38:36.527118 14788 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:36.522319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:43741 (local address 127.5.122.190:38593)
0620 11:38:36.522423 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:36.522425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:36.522524 (+ 99us) server_negotiation.cc:407] Connection header received
0620 11:38:36.522608 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:36.522609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:36.522620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:36.522668 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:36.523003 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.523517 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.524315 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.524534 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.524620 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:36.527052 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:43741: BlockingRecv error: recv got EOF from 127.5.122.130:43741 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 20 11:38:36 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:36 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:36.648407 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:36.643880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:50689)
0620 11:38:36.644045 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:36.644056 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:36.644140 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:36.644257 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:36.644258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:36.644433 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:36.644479 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:36.644480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.645119 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.645119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:36.645641 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:36.645642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.646038 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.646039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:36.646102 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:36.646537 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:36.646542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:36.648347 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
W20260620 11:38:36.648541 14788 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:36.643976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:50689 (local address 127.5.122.190:38593)
0620 11:38:36.644046 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:36.644048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:36.644150 (+ 102us) server_negotiation.cc:407] Connection header received
0620 11:38:36.644181 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:36.644182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:36.644194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:36.644243 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:36.644534 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.645031 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.645725 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:36.645911 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:36.646034 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:36.648455 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:50689: BlockingRecv error: recv got EOF from 127.5.122.131:50689 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 20 11:38:37 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:37 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:37.387483 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:37.382371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:51293)
0620 11:38:37.382508 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:37.382527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:37.382579 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:37.382774 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:37.382775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:37.382961 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:37.383013 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:37.383015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.383670 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.383671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:37.384247 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:37.384249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.384606 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.384607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:37.384710 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:37.385351 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:37.385364 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:37.387409 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260620 11:38:37.387586 14792 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:37.382492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:51293 (local address 127.5.122.190:38593)
0620 11:38:37.382637 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:37.382638 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:37.382647 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:37.382672 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:37.382672 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:37.382684 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:37.382756 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:37.383086 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.383599 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.384339 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.384534 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.384588 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:37.387513 (+ 2925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:51293: BlockingRecv error: recv got EOF from 127.5.122.129:51293 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
Jun 20 11:38:37 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:37 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:37.532284 14793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:37.527674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:55741)
0620 11:38:37.527795 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:37.527808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:37.527890 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:37.528072 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:37.528072 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:37.528245 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:37.528295 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:37.528296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.528938 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.528939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:37.529471 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:37.529473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.529822 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.529823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:37.529862 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:37.530340 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:37.530344 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:37.532222 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
W20260620 11:38:37.532490 14792 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:37.527768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:55741 (local address 127.5.122.190:38593)
0620 11:38:37.527821 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:37.527823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:37.527855 (+ 32us) server_negotiation.cc:407] Connection header received
0620 11:38:37.527979 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:37.527980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:37.527992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:37.528051 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:37.528364 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.528858 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.529558 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.529753 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.529809 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:37.532428 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:55741: BlockingRecv error: recv got EOF from 127.5.122.130:55741 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 20 11:38:37 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:37 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:37.653661 14794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:37.648861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:49391)
0620 11:38:37.649042 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:37.649060 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:37.649153 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:37.649303 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:37.649304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:37.649497 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:37.649544 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:37.649547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.650273 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.650274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:37.650830 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:37.650831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.651264 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.651265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:37.651344 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:37.651790 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:37.651795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:37.653591 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
W20260620 11:38:37.653771 14792 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:37.648922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:49391 (local address 127.5.122.190:38593)
0620 11:38:37.648990 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:37.648996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:37.649133 (+ 137us) server_negotiation.cc:407] Connection header received
0620 11:38:37.649211 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:37.649212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:37.649227 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:37.649287 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:37.649611 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.650192 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.650905 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:37.651112 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:37.651183 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:37.653695 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:49391: BlockingRecv error: recv got EOF from 127.5.122.131:49391 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 20 11:38:38 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:38 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:38.393047 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:38.388062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:39017)
0620 11:38:38.388216 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:38.388233 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:38.388298 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:38.388520 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:38.388521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:38.388761 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:38.388811 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:38.388813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.389581 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.389583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:38.390217 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:38.390220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.390592 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.390593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:38.390647 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:38.391105 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:38.391110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:38.392947 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
W20260620 11:38:38.393117 14796 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:38.388109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:39017 (local address 127.5.122.190:38593)
0620 11:38:38.388242 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:38.388244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:38.388297 (+ 53us) server_negotiation.cc:407] Connection header received
0620 11:38:38.388416 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:38.388418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:38.388428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:38.388489 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:38.388892 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.389482 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.390288 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.390493 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.390547 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:38.393033 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:39017: BlockingRecv error: recv got EOF from 127.5.122.129:39017 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
Jun 20 11:38:38 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:38 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:38.542992 14797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:38.532822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:46199)
0620 11:38:38.532997 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:38.533008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:38.533067 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:38.533198 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:38.533199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:38.533384 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:38.533437 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:38.533439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.534133 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.534134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:38.534744 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:38.534745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.535201 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.535202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:38.535251 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:38.540714 (+ 5463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:38.540720 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:38.542921 (+ 2201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
W20260620 11:38:38.543130 14796 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:38.532890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:46199 (local address 127.5.122.190:38593)
0620 11:38:38.532969 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:38.532971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:38.533066 (+ 95us) server_negotiation.cc:407] Connection header received
0620 11:38:38.533124 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:38.533125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:38.533137 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:38.533184 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:38.533498 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.534054 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.534897 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.535118 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.535185 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:38.543035 (+ 7850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:46199: BlockingRecv error: recv got EOF from 127.5.122.130:46199 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 20 11:38:38 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:38 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.131@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:38.659204 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:38.654242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.131:33423)
0620 11:38:38.654432 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:38.654445 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:38.654518 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:38.654661 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:38.654662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:38.654840 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:38.654888 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:38.654890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.655603 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.655604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:38.656156 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:38.656158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.656507 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.656508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:38.656551 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:38.657062 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:38.657067 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:38.659127 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":104,"threads_started":1}
W20260620 11:38:38.659351 14796 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:38.654296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:33423 (local address 127.5.122.190:38593)
0620 11:38:38.654353 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:38.654356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:38.654522 (+ 166us) server_negotiation.cc:407] Connection header received
0620 11:38:38.654575 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:38.654576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:38.654589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:38.654648 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:38.654985 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.655510 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.656252 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:38.656439 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:38.656494 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:38.659260 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.131:33423: BlockingRecv error: recv got EOF from 127.5.122.131:33423 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 20 11:38:39 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:39 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.129@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:39.398710 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:39.393703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.129:43171)
0620 11:38:39.393879 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:39.393890 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:39.393944 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:39.394188 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:39.394188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:39.394356 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:39.394423 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:39.394425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.395092 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.395093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:39.395618 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:39.395620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.395957 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.395958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:39.396001 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:39.396542 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:39.396547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:39.398647 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
W20260620 11:38:39.398821 14800 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:39.393770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:43171 (local address 127.5.122.190:38593)
0620 11:38:39.393936 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:39.393938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:39.393950 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:38:39.394083 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:39.394084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:39.394095 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:39.394167 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:39.394494 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.395013 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.395696 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.395888 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.395951 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:39.398742 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.129:43171: BlockingRecv error: recv got EOF from 127.5.122.129:43171 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
Jun 20 11:38:39 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
Jun 20 11:38:39 dist-test-slave-szdr krb5kdc[13741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.122.130@KRBTEST.COM for oryx/127.5.122.190@KRBTEST.COM, Server not found in Kerberos database
I20260620 11:38:39.548502 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:39.543556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38593 (local address 127.5.122.130:54101)
0620 11:38:39.543732 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:39.543745 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:39.543849 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:39.544028 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:39.544029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:39.544219 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:39.544274 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:39.544276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.545032 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.545033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:39.545579 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:39.545581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.546012 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.546013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:39.546080 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:39.546548 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:39.546552 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:39.548401 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.122.190:38593: Server oryx/127.5.122.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
W20260620 11:38:39.548564 14800 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:39.543667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:54101 (local address 127.5.122.190:38593)
0620 11:38:39.543745 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:39.543748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:39.543826 (+ 78us) server_negotiation.cc:407] Connection header received
0620 11:38:39.543951 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:39.543951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:39.543964 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:39.544013 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:39.544371 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.544923 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.545673 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.545884 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.545972 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:39.548501 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.122.130:54101: BlockingRecv error: recv got EOF from 127.5.122.130:54101 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 11:38:39.573222 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 14324
I20260620 11:38:39.578725 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 14476
I20260620 11:38:39.583925 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 14610
I20260620 11:38:39.588922 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 14254
2026-06-20T11:38:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17466 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T11:38:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:39Z Disabled control of system clock
I20260620 11:38:39.618471 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41181
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 11:38:39.701735 14805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:39.702008 14805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:39.702033 14805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:39.703256 14805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:39.703297 14805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:39.703310 14805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:39.703323 14805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:39.703337 14805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:39.704694 14805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:39.704909 14805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:39.705211 14805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:38:39.711059 14805 server_base.cc:1034] running on GCE node
W20260620 11:38:39.711078 14810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:39.711184 14813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:39.711192 14811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:39.711751 14805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:39.712113 14805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:39.713291 14805 hybrid_clock.cc:648] HybridClock initialized: now 1781955519713271 us; error 42 us; skew 500 ppm
I20260620 11:38:39.714802 14805 webserver.cc:460] Webserver started at http://127.5.122.190:42241/ using document root <none> and password file <none>
I20260620 11:38:39.715099 14805 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:39.715152 14805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:39.715345 14805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:39.716588 14805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "63b14bbdc46242db964b38cb39bf83d5"
format_stamp: "Formatted at 2026-06-20 11:38:39 on dist-test-slave-szdr"
server_key: "610345f24ad2524f39bc1af117eac8af"
server_key_iv: "d91fef2745dd8d3ab9c50d66d259e9d5"
server_key_version: "encryptionkey@0"
I20260620 11:38:39.717000 14805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "63b14bbdc46242db964b38cb39bf83d5"
format_stamp: "Formatted at 2026-06-20 11:38:39 on dist-test-slave-szdr"
server_key: "610345f24ad2524f39bc1af117eac8af"
server_key_iv: "d91fef2745dd8d3ab9c50d66d259e9d5"
server_key_version: "encryptionkey@0"
I20260620 11:38:39.718559 14805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260620 11:38:39.719476 14819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:39.719696 14805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:39.719792 14805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "63b14bbdc46242db964b38cb39bf83d5"
format_stamp: "Formatted at 2026-06-20 11:38:39 on dist-test-slave-szdr"
server_key: "610345f24ad2524f39bc1af117eac8af"
server_key_iv: "d91fef2745dd8d3ab9c50d66d259e9d5"
server_key_version: "encryptionkey@0"
I20260620 11:38:39.719908 14805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:39.733765 14805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:39.734225 14805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:39.734395 14805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:39.738615 14805 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:41181
I20260620 11:38:39.738652 14871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:41181 every 8 connection(s)
I20260620 11:38:39.739060 14805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:39.739828 14872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:39.742894 14872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5: Bootstrap starting.
I20260620 11:38:39.743188 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 14805
I20260620 11:38:39.743302 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:39.743533 5610 external_mini_cluster.cc:1428] Setting key 4b296fd860f87865139630db3dc0e285
I20260620 11:38:39.743701 14872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:39.744123 14872 log.cc:824] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:39.744892 14872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5: No bootstrap required, opened a new log
I20260620 11:38:39.746660 14872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b14bbdc46242db964b38cb39bf83d5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41181 } }
I20260620 11:38:39.746860 14872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:39.746897 14872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b14bbdc46242db964b38cb39bf83d5, State: Initialized, Role: FOLLOWER
I20260620 11:38:39.747066 14872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b14bbdc46242db964b38cb39bf83d5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41181 } }
I20260620 11:38:39.747166 14872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:39.747196 14872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:39.747242 14872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:39.748383 14872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b14bbdc46242db964b38cb39bf83d5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41181 } }
I20260620 11:38:39.748518 14872 leader_election.cc:302] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63b14bbdc46242db964b38cb39bf83d5; no voters:
I20260620 11:38:39.748742 14872 leader_election.cc:288] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:39.748772 14877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:39.748889 14877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [term 1 LEADER]: Becoming Leader. State: Replica: 63b14bbdc46242db964b38cb39bf83d5, State: Running, Role: LEADER
I20260620 11:38:39.748970 14877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b14bbdc46242db964b38cb39bf83d5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41181 } }
I20260620 11:38:39.749050 14872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:39.749308 14877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63b14bbdc46242db964b38cb39bf83d5. Latest consensus state: current_term: 1 leader_uuid: "63b14bbdc46242db964b38cb39bf83d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b14bbdc46242db964b38cb39bf83d5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41181 } } }
I20260620 11:38:39.749344 14878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63b14bbdc46242db964b38cb39bf83d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b14bbdc46242db964b38cb39bf83d5" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41181 } } }
I20260620 11:38:39.749480 14877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:39.749495 14878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:39.749760 14883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:39.750154 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:39.744144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.5.122.190:41181)
0620 11:38:39.744390 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:39.744398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:39.744425 (+ 27us) server_negotiation.cc:407] Connection header received
0620 11:38:39.744881 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:39.744889 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:39.744997 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:39.745153 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:39.745437 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.746490 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.747589 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.748186 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.748288 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:39.749126 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:39.749130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:39.749130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:39.749369 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:39.749417 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:39.749433 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:39.749779 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:39.749818 (+ 39us) server_negotiation.cc:299] Negotiation successful
0620 11:38:39.749904 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260620 11:38:39.750512 14883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:39.752203 14883 catalog_manager.cc:1347] Generated new cluster ID: 0a213bbc55ae4ae092ed50a2e4cb4e28
I20260620 11:38:39.752264 14883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:39.767731 14883 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:39.768482 14883 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:39.774245 14883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63b14bbdc46242db964b38cb39bf83d5: Generated new TSK 0
I20260620 11:38:39.774506 14883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:38:39.778532 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41181
--builtin_ntp_servers=127.5.122.148:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 11:38:39.863057 14896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:39.863382 14896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:39.863418 14896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:39.864745 14896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:39.864801 14896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:39.864830 14896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:39.866379 14896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:39.866688 14896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:39.867024 14896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:39.872450 14901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:39.872450 14902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:39.872454 14904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:39.873044 14896 server_base.cc:1034] running on GCE node
I20260620 11:38:39.873296 14896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:39.873588 14896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:39.874774 14896 hybrid_clock.cc:648] HybridClock initialized: now 1781955519874744 us; error 46 us; skew 500 ppm
I20260620 11:38:39.876435 14896 webserver.cc:460] Webserver started at http://127.5.122.129:42781/ using document root <none> and password file <none>
I20260620 11:38:39.876727 14896 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:39.876806 14896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:39.876992 14896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:39.878332 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "0b4a9962f04541be88bfa2a56cfcd6b5"
format_stamp: "Formatted at 2026-06-20 11:38:39 on dist-test-slave-szdr"
server_key: "18dc98c9803c6e8771d88000c33cdaef"
server_key_iv: "3c6c6a661389fa3059280b961ed5fbaa"
server_key_version: "encryptionkey@0"
I20260620 11:38:39.878746 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "0b4a9962f04541be88bfa2a56cfcd6b5"
format_stamp: "Formatted at 2026-06-20 11:38:39 on dist-test-slave-szdr"
server_key: "18dc98c9803c6e8771d88000c33cdaef"
server_key_iv: "3c6c6a661389fa3059280b961ed5fbaa"
server_key_version: "encryptionkey@0"
I20260620 11:38:39.880342 14896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:39.881217 14910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:39.881516 14896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:39.881623 14896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "0b4a9962f04541be88bfa2a56cfcd6b5"
format_stamp: "Formatted at 2026-06-20 11:38:39 on dist-test-slave-szdr"
server_key: "18dc98c9803c6e8771d88000c33cdaef"
server_key_iv: "3c6c6a661389fa3059280b961ed5fbaa"
server_key_version: "encryptionkey@0"
I20260620 11:38:39.881707 14896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:39.897192 14896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:39.897514 14896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:39.897643 14896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:39.897861 14896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:39.898283 14896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:39.898320 14896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:39.898384 14896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:39.898411 14896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:39.904399 14896 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:43129
I20260620 11:38:39.904507 15023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:43129 every 8 connection(s)
I20260620 11:38:39.904917 14896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:39.910785 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:39.905795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:49425 (local address 127.5.122.190:41181)
0620 11:38:39.905894 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:39.905911 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:39.906194 (+ 283us) server_negotiation.cc:407] Connection header received
0620 11:38:39.906783 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:39.906784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:39.906797 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:39.906867 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:39.907619 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.908135 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.909200 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.909404 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.909482 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:39.910459 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:39.910461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:39.910461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:39.910468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:39.910484 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:39.910489 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:39.910567 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:39.910666 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 11:38:39.910698 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 11:38:39.911054 15026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:39.905806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41181 (local address 127.5.122.129:49425)
0620 11:38:39.906058 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:39.906100 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:39.906667 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:39.906949 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:39.906954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:39.906970 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:39.907511 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:39.907516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.908243 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.908245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:39.909096 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:39.909098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:39.909504 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:39.909505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:39.909655 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:39.910295 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:39.910296 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:39.910330 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:39.910357 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:39.910568 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:39.910575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:39.910578 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:39.910669 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 11:38:39.910786 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260620 11:38:39.911644 15024 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41181
I20260620 11:38:39.911767 15024 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:39.911978 15024 heartbeater.cc:511] Master 127.5.122.190:41181 requested a full tablet report, sending...
I20260620 11:38:39.912503 14836 ts_manager.cc:194] Registered new tserver with Master: 0b4a9962f04541be88bfa2a56cfcd6b5 (127.5.122.129:43129)
I20260620 11:38:39.913162 14836 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.129:49425
I20260620 11:38:39.914170 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 14896
I20260620 11:38:39.914251 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:39.914463 5610 external_mini_cluster.cc:1428] Setting key 32f6b2e3aa1644ad5bf2aa2ae916f0c5
I20260620 11:38:39.915444 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41181
--builtin_ntp_servers=127.5.122.148:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 11:38:40.000536 15027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:40.000825 15027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:40.000865 15027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:40.002125 15027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:40.002206 15027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:40.002247 15027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:40.003700 15027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:40.004032 15027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:40.004376 15027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:38:40.010324 15027 server_base.cc:1034] running on GCE node
W20260620 11:38:40.010299 15032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.010401 15033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.010304 15035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.010766 15027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:40.011021 15027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:40.012169 15027 hybrid_clock.cc:648] HybridClock initialized: now 1781955520012156 us; error 36 us; skew 500 ppm
I20260620 11:38:40.013522 15027 webserver.cc:460] Webserver started at http://127.5.122.130:33259/ using document root <none> and password file <none>
I20260620 11:38:40.013746 15027 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:40.013792 15027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:40.013890 15027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:40.015056 15027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "fdbca46c3e33430994abd00517848319"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "f93919c1f61d8e4297fedf703d9cf6ea"
server_key_iv: "908548bd81bcd945841256ff21394123"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.015372 15027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "fdbca46c3e33430994abd00517848319"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "f93919c1f61d8e4297fedf703d9cf6ea"
server_key_iv: "908548bd81bcd945841256ff21394123"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.016779 15027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:38:40.017513 15041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.017690 15027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:40.017786 15027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "fdbca46c3e33430994abd00517848319"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "f93919c1f61d8e4297fedf703d9cf6ea"
server_key_iv: "908548bd81bcd945841256ff21394123"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.017848 15027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:40.028077 15027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:40.028357 15027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:40.028462 15027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:40.028672 15027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:40.029053 15027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:40.029093 15027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.029139 15027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:40.029165 15027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.034932 15027 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:43645
I20260620 11:38:40.034976 15154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:43645 every 8 connection(s)
I20260620 11:38:40.035349 15027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:40.040706 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.036122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:53469 (local address 127.5.122.190:41181)
0620 11:38:40.036195 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.036197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.036425 (+ 228us) server_negotiation.cc:407] Connection header received
0620 11:38:40.036929 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.036930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.036940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.037005 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.037752 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.038338 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.039361 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.039546 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.039602 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:40.040407 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:40.040409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:40.040409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:40.040414 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:40.040429 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:40.040432 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:40.040475 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:40.040616 (+ 141us) server_negotiation.cc:299] Negotiation successful
0620 11:38:40.040640 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 11:38:40.040742 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15027
I20260620 11:38:40.040836 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:40.041009 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.036081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41181 (local address 127.5.122.130:53469)
0620 11:38:40.036280 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:40.036325 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:40.036844 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:40.037126 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:40.037130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:40.037139 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:40.037675 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:40.037680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.038428 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.038428 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:40.039281 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:40.039283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.039621 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.039622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:40.039731 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:40.040270 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:40.040271 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:40.040300 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:40.040323 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:40.040487 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:40.040491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:40.040494 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:40.040616 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 11:38:40.040745 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260620 11:38:40.041049 5610 external_mini_cluster.cc:1428] Setting key d31333ebdc37a468bdd4f55a17b6dcc0
I20260620 11:38:40.041589 15155 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41181
I20260620 11:38:40.041706 15155 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:40.041919 15155 heartbeater.cc:511] Master 127.5.122.190:41181 requested a full tablet report, sending...
I20260620 11:38:40.042145 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41181
--builtin_ntp_servers=127.5.122.148:43113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 11:38:40.042374 14836 ts_manager.cc:194] Registered new tserver with Master: fdbca46c3e33430994abd00517848319 (127.5.122.130:43645)
I20260620 11:38:40.042876 14836 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.130:53469
W20260620 11:38:40.126609 15158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:40.126891 15158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:40.126915 15158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:40.128087 15158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:40.128124 15158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:40.128149 15158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:40.129508 15158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:40.129712 15158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:40.129978 15158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:40.135145 15163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.135641 15166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.135691 15164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.135675 15158 server_base.cc:1034] running on GCE node
I20260620 11:38:40.135967 15158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:40.136229 15158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:40.137396 15158 hybrid_clock.cc:648] HybridClock initialized: now 1781955520137379 us; error 33 us; skew 500 ppm
I20260620 11:38:40.138839 15158 webserver.cc:460] Webserver started at http://127.5.122.131:46055/ using document root <none> and password file <none>
I20260620 11:38:40.139108 15158 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:40.139181 15158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:40.139333 15158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:40.140426 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "e3ae3e1340194406814ec85b981002c2"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "753a9a34cf0fa305db2fa0f95a9d442e"
server_key_iv: "5344cfb7fcf977e8eefd3df2030fb220"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.140803 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "e3ae3e1340194406814ec85b981002c2"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "753a9a34cf0fa305db2fa0f95a9d442e"
server_key_iv: "5344cfb7fcf977e8eefd3df2030fb220"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.142369 15158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:40.143123 15172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.143301 15158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:40.143414 15158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "e3ae3e1340194406814ec85b981002c2"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "753a9a34cf0fa305db2fa0f95a9d442e"
server_key_iv: "5344cfb7fcf977e8eefd3df2030fb220"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.143503 15158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:40.167158 15158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:40.167541 15158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:40.167718 15158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:40.167968 15158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:40.168402 15158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:40.168457 15158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.168493 15158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:40.168560 15158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.175410 15158 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:37607
I20260620 11:38:40.175447 15285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:37607 every 8 connection(s)
I20260620 11:38:40.175820 15158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:40.177698 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15158
I20260620 11:38:40.177809 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:40.178051 5610 external_mini_cluster.cc:1428] Setting key 5f10b01ee525892ff1058ad370b76e04
I20260620 11:38:40.181485 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.176649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:37179 (local address 127.5.122.190:41181)
0620 11:38:40.176691 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.176693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.176953 (+ 260us) server_negotiation.cc:407] Connection header received
0620 11:38:40.177454 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.177454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.177464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.177517 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.178523 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.179202 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.180195 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.180376 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.180428 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:40.181246 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:40.181247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:40.181247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:40.181252 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:40.181266 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:40.181270 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:40.181312 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:40.181397 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 11:38:40.181421 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 11:38:40.181697 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.176613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41181 (local address 127.5.122.131:37179)
0620 11:38:40.176815 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:40.176858 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:40.177369 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:40.177602 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:40.177606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:40.177618 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:40.178391 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:40.178420 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.179282 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.179283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:40.180109 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:40.180112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:40.180450 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:40.180451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:40.180568 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:40.181107 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:40.181107 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:40.181139 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:40.181163 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:40.181327 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:40.181331 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:40.181333 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:40.181386 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 11:38:40.181474 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":102,"threads_started":1}
I20260620 11:38:40.182211 15286 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41181
I20260620 11:38:40.182305 15286 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:40.182456 15286 heartbeater.cc:511] Master 127.5.122.190:41181 requested a full tablet report, sending...
I20260620 11:38:40.182809 14836 ts_manager.cc:194] Registered new tserver with Master: e3ae3e1340194406814ec85b981002c2 (127.5.122.131:37607)
I20260620 11:38:40.183152 14836 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.131:37179
I20260620 11:38:40.190095 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 11:38:40.192072 14875 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:40.191494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.5.122.190:41181)
0620 11:38:40.191560 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.191562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.191708 (+ 146us) server_negotiation.cc:407] Connection header received
0620 11:38:40.191750 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.191751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.191761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.191802 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.191978 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46748: BlockingRecv error: recv got EOF from 127.0.0.1:46748 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 11:38:40.192088 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 14896
I20260620 11:38:40.196949 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15027
I20260620 11:38:40.201582 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15158
I20260620 11:38:40.205950 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 14805
2026-06-20T11:38:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (615 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T11:38:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:40Z Disabled control of system clock
I20260620 11:38:40.233568 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:32867
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:32867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 11:38:40.315943 15297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:40.316218 15297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:40.316255 15297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:40.317483 15297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:40.317564 15297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:40.317586 15297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:40.317622 15297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:40.317674 15297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:40.319136 15297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:32867
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:32867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:40.319428 15297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:40.319741 15297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:40.324738 15305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.324738 15302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.324800 15303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.324754 15297 server_base.cc:1034] running on GCE node
I20260620 11:38:40.325186 15297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:40.325470 15297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:40.326644 15297 hybrid_clock.cc:648] HybridClock initialized: now 1781955520326631 us; error 32 us; skew 500 ppm
I20260620 11:38:40.327903 15297 webserver.cc:460] Webserver started at http://127.5.122.190:36917/ using document root <none> and password file <none>
I20260620 11:38:40.328145 15297 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:40.328215 15297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:40.328365 15297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:40.329461 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "e6088e4c20cf4547b12eb1f11f430252"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "40dd2278f91909bb4467291e4a143623"
server_key_iv: "9b9296919ddbd43a698a9a87f1c5fcad"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.329855 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "e6088e4c20cf4547b12eb1f11f430252"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "40dd2278f91909bb4467291e4a143623"
server_key_iv: "9b9296919ddbd43a698a9a87f1c5fcad"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.331192 15297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:40.331945 15311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.332113 15297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260620 11:38:40.332223 15297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "e6088e4c20cf4547b12eb1f11f430252"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "40dd2278f91909bb4467291e4a143623"
server_key_iv: "9b9296919ddbd43a698a9a87f1c5fcad"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.332297 15297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:40.335801 15297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:40.336081 15297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:40.336229 15297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:40.340104 15297 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:32867
I20260620 11:38:40.340137 15363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:32867 every 8 connection(s)
I20260620 11:38:40.340487 15297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:40.341198 15364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:40.344111 15364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252: Bootstrap starting.
I20260620 11:38:40.344724 15364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:40.345044 15364 log.cc:824] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:40.345718 15364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252: No bootstrap required, opened a new log
I20260620 11:38:40.347443 15364 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6088e4c20cf4547b12eb1f11f430252" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 32867 } }
I20260620 11:38:40.347591 15364 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:40.347615 15364 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6088e4c20cf4547b12eb1f11f430252, State: Initialized, Role: FOLLOWER
I20260620 11:38:40.347800 15364 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6088e4c20cf4547b12eb1f11f430252" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 32867 } }
I20260620 11:38:40.347877 15364 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:40.347900 15364 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:40.347927 15364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:40.348269 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15297
I20260620 11:38:40.348392 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:40.348593 5610 external_mini_cluster.cc:1428] Setting key 6af70852d33323916e4d0334603e1c09
I20260620 11:38:40.348618 15364 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6088e4c20cf4547b12eb1f11f430252" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 32867 } }
I20260620 11:38:40.348732 15364 leader_election.cc:302] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6088e4c20cf4547b12eb1f11f430252; no voters:
I20260620 11:38:40.348913 15364 leader_election.cc:288] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:40.348944 15367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:40.349220 15364 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:40.349352 15367 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [term 1 LEADER]: Becoming Leader. State: Replica: e6088e4c20cf4547b12eb1f11f430252, State: Running, Role: LEADER
I20260620 11:38:40.349439 15367 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6088e4c20cf4547b12eb1f11f430252" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 32867 } }
I20260620 11:38:40.350087 15368 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6088e4c20cf4547b12eb1f11f430252" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6088e4c20cf4547b12eb1f11f430252" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 32867 } } }
I20260620 11:38:40.350186 15368 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:40.350576 15370 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6088e4c20cf4547b12eb1f11f430252. Latest consensus state: current_term: 1 leader_uuid: "e6088e4c20cf4547b12eb1f11f430252" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6088e4c20cf4547b12eb1f11f430252" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 32867 } } }
I20260620 11:38:40.350734 15370 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252 [sys.catalog]: This master's current role is: LEADER
W20260620 11:38:40.351354 15386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:38:40.351421 15386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:38:40.351598 15379 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:40.351626 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.349242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.5.122.190:32867)
0620 11:38:40.349633 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.349642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.349675 (+ 33us) server_negotiation.cc:407] Connection header received
0620 11:38:40.349804 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.349812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.349869 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.350384 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.351075 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:40.351079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:40.351080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:40.351084 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:40.351123 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:40.351139 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:40.351207 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:40.351325 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 11:38:40.351387 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":61,"threads_started":1}
I20260620 11:38:40.351790 15379 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:40.353327 15379 catalog_manager.cc:1347] Generated new cluster ID: d1eb02bdad9f4b3089ea6bef9973c875
I20260620 11:38:40.353379 15379 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:40.364770 15379 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:40.365010 15379 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:40.373075 15379 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6088e4c20cf4547b12eb1f11f430252: Generated new TSK 0
I20260620 11:38:40.373314 15379 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:38:40.380388 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:32867
--builtin_ntp_servers=127.5.122.148:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 11:38:40.469045 15389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:40.469280 15389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:40.469305 15389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:40.470729 15389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:40.470778 15389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:40.470808 15389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:40.472335 15389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:32867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:40.472630 15389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:40.473014 15389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:40.479346 15397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.479347 15395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.479698 15389 server_base.cc:1034] running on GCE node
W20260620 11:38:40.479831 15394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.480093 15389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:40.480487 15389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:40.481688 15389 hybrid_clock.cc:648] HybridClock initialized: now 1781955520481678 us; error 37 us; skew 500 ppm
I20260620 11:38:40.483275 15389 webserver.cc:460] Webserver started at http://127.5.122.129:42229/ using document root <none> and password file <none>
I20260620 11:38:40.483599 15389 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:40.483667 15389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:40.483817 15389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:40.485258 15389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "09cd4f0cd4ba4184806e33fe2d7893ff"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "9ec6d8f89e37497509a2ab0077511568"
server_key_iv: "907f733f9ba41a645b87ba0631a376c5"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.485616 15389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "09cd4f0cd4ba4184806e33fe2d7893ff"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "9ec6d8f89e37497509a2ab0077511568"
server_key_iv: "907f733f9ba41a645b87ba0631a376c5"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.487146 15389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:38:40.487954 15403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.488248 15389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:38:40.488358 15389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "09cd4f0cd4ba4184806e33fe2d7893ff"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "9ec6d8f89e37497509a2ab0077511568"
server_key_iv: "907f733f9ba41a645b87ba0631a376c5"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.488443 15389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:40.492193 15389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:40.492501 15389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:40.492656 15389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:40.492892 15389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:40.493212 15389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:40.493247 15389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.493299 15389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:40.493324 15389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.499305 15389 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:46197
I20260620 11:38:40.499352 15516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:46197 every 8 connection(s)
I20260620 11:38:40.499697 15389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:40.502807 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.500496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:39159 (local address 127.5.122.190:32867)
0620 11:38:40.500541 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.500543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.500797 (+ 254us) server_negotiation.cc:407] Connection header received
0620 11:38:40.501268 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.501269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.501275 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.501336 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.502529 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:40.502530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:40.502530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:40.502535 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:40.502553 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:40.502557 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:40.502626 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:40.502731 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 11:38:40.502749 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:38:40.502969 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.500467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:32867 (local address 127.5.122.129:39159)
0620 11:38:40.500661 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:40.500704 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:40.501185 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:40.501389 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:40.501393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:40.501402 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:40.502361 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:40.502362 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:40.502392 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:40.502424 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:40.502627 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:40.502638 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:40.502641 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:40.502699 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 11:38:40.502750 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":106,"threads_started":1}
I20260620 11:38:40.503484 15517 heartbeater.cc:347] Connected to a master server at 127.5.122.190:32867
I20260620 11:38:40.503568 15517 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:40.503750 15517 heartbeater.cc:511] Master 127.5.122.190:32867 requested a full tablet report, sending...
I20260620 11:38:40.504293 15328 ts_manager.cc:194] Registered new tserver with Master: 09cd4f0cd4ba4184806e33fe2d7893ff (127.5.122.129:46197)
I20260620 11:38:40.509294 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15389
I20260620 11:38:40.509377 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:40.509589 5610 external_mini_cluster.cc:1428] Setting key b4ecf2d2b41d635f2388812a5d7b3f42
I20260620 11:38:40.510643 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:32867
--builtin_ntp_servers=127.5.122.148:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 11:38:40.593297 15520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:40.593554 15520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:40.593593 15520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:40.594918 15520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:40.594988 15520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:40.595034 15520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:40.596446 15520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:32867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:40.596736 15520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:40.597044 15520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:40.602190 15525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.602360 15520 server_base.cc:1034] running on GCE node
W20260620 11:38:40.602257 15526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.602195 15528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.602692 15520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:40.602953 15520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:40.604131 15520 hybrid_clock.cc:648] HybridClock initialized: now 1781955520604118 us; error 42 us; skew 500 ppm
I20260620 11:38:40.605489 15520 webserver.cc:460] Webserver started at http://127.5.122.130:40759/ using document root <none> and password file <none>
I20260620 11:38:40.605759 15520 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:40.605834 15520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:40.606048 15520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:40.607172 15520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "1212ab8539aa48ec9f739ac494d6a0ed"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "886e33c24b67a175ac8c953c0369bcc1"
server_key_iv: "e0631988f489460d218f7c14bbc765de"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.607559 15520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "1212ab8539aa48ec9f739ac494d6a0ed"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "886e33c24b67a175ac8c953c0369bcc1"
server_key_iv: "e0631988f489460d218f7c14bbc765de"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.608925 15520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:40.609644 15534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.609848 15520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:40.609952 15520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "1212ab8539aa48ec9f739ac494d6a0ed"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "886e33c24b67a175ac8c953c0369bcc1"
server_key_iv: "e0631988f489460d218f7c14bbc765de"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.610069 15520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:40.613554 15520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:40.613838 15520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:40.614032 15520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:40.614261 15520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:40.614579 15520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:40.614634 15520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.614674 15520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:40.614701 15520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.620560 15520 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:40365
I20260620 11:38:40.620599 15647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:40365 every 8 connection(s)
I20260620 11:38:40.620972 15520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:38:40.624001 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.621806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:59033 (local address 127.5.122.190:32867)
0620 11:38:40.621876 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.621879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.622126 (+ 247us) server_negotiation.cc:407] Connection header received
0620 11:38:40.622617 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.622618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.622623 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.622684 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.623804 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:40.623805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:40.623805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:40.623809 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:40.623825 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:40.623829 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:40.623877 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:40.623930 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 11:38:40.623943 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 11:38:40.624246 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.621782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:32867 (local address 127.5.122.130:59033)
0620 11:38:40.622000 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:40.622044 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:40.622524 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:40.622749 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:40.622752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:40.622760 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:40.623638 (+ 878us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:40.623638 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:40.623671 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:40.623701 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:40.623873 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:40.623876 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:40.623878 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:38:40.623931 (+ 53us) client_negotiation.cc:240] Negotiation successful
0620 11:38:40.623983 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260620 11:38:40.624835 15648 heartbeater.cc:347] Connected to a master server at 127.5.122.190:32867
I20260620 11:38:40.624925 15648 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:40.625101 15648 heartbeater.cc:511] Master 127.5.122.190:32867 requested a full tablet report, sending...
I20260620 11:38:40.625267 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15520
I20260620 11:38:40.625375 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:40.625576 15328 ts_manager.cc:194] Registered new tserver with Master: 1212ab8539aa48ec9f739ac494d6a0ed (127.5.122.130:40365)
I20260620 11:38:40.625610 5610 external_mini_cluster.cc:1428] Setting key a24419e8614d8b5f86a6bf16294396eb
I20260620 11:38:40.626536 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:32867
--builtin_ntp_servers=127.5.122.148:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 11:38:40.709120 15651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:40.709432 15651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:40.709486 15651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:40.710840 15651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:40.710937 15651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:40.710985 15651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:40.712502 15651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:32867
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:40.712888 15651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:40.713238 15651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:40.718523 15657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.718582 15656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:40.718523 15659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:40.718891 15651 server_base.cc:1034] running on GCE node
I20260620 11:38:40.719182 15651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:40.719477 15651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:40.720664 15651 hybrid_clock.cc:648] HybridClock initialized: now 1781955520720645 us; error 57 us; skew 500 ppm
I20260620 11:38:40.722141 15651 webserver.cc:460] Webserver started at http://127.5.122.131:39071/ using document root <none> and password file <none>
I20260620 11:38:40.722445 15651 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:40.722519 15651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:40.722664 15651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:40.724088 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "282edaf0cf7548589c4d6dbc422470e1"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "49b694ccb86174b661b6a3787a7fdaea"
server_key_iv: "0c2ad000a9e3e2b85708a90b3bcce465"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.724570 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "282edaf0cf7548589c4d6dbc422470e1"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "49b694ccb86174b661b6a3787a7fdaea"
server_key_iv: "0c2ad000a9e3e2b85708a90b3bcce465"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.726691 15651 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260620 11:38:40.730134 15665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.731379 15651 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260620 11:38:40.731526 15651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "282edaf0cf7548589c4d6dbc422470e1"
format_stamp: "Formatted at 2026-06-20 11:38:40 on dist-test-slave-szdr"
server_key: "49b694ccb86174b661b6a3787a7fdaea"
server_key_iv: "0c2ad000a9e3e2b85708a90b3bcce465"
server_key_version: "encryptionkey@0"
I20260620 11:38:40.731662 15651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:40.736104 15651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:40.736419 15651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:40.736572 15651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:40.736802 15651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:40.737146 15651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:40.737210 15651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.737252 15651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:40.737280 15651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:40.743251 15651 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:35265
I20260620 11:38:40.743340 15778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:35265 every 8 connection(s)
I20260620 11:38:40.743728 15651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:40.746937 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.744540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:53441 (local address 127.5.122.190:32867)
0620 11:38:40.744621 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.744623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.744889 (+ 266us) server_negotiation.cc:407] Connection header received
0620 11:38:40.745374 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.745375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.745380 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.745448 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.746709 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:40.746711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:40.746712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:38:40.746715 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:40.746729 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:38:40.746734 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:40.746794 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:40.746865 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 11:38:40.746880 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 11:38:40.747112 15781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:40.744526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:32867 (local address 127.5.122.131:53441)
0620 11:38:40.744749 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:40.744794 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:40.745282 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:40.745505 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:40.745508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:40.745517 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:40.746550 (+ 1033us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:38:40.746551 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:40.746582 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:38:40.746615 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:40.746798 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:40.746801 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:40.746804 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:40.746858 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:38:40.746909 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
I20260620 11:38:40.747594 15779 heartbeater.cc:347] Connected to a master server at 127.5.122.190:32867
I20260620 11:38:40.747686 15779 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:40.747807 15779 heartbeater.cc:511] Master 127.5.122.190:32867 requested a full tablet report, sending...
I20260620 11:38:40.748206 15328 ts_manager.cc:194] Registered new tserver with Master: 282edaf0cf7548589c4d6dbc422470e1 (127.5.122.131:35265)
I20260620 11:38:40.751300 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15651
I20260620 11:38:40.751427 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:40.751684 5610 external_mini_cluster.cc:1428] Setting key 639cbee6924b5e9c4b9c89525055f0c0
I20260620 11:38:40.753615 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 11:38:40.755872 15371 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:38:40.755151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.5.122.190:32867)
0620 11:38:40.755202 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:40.755205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:40.755360 (+ 155us) server_negotiation.cc:407] Connection header received
0620 11:38:40.755450 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:40.755451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:40.755456 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:40.755561 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:40.755802 (+ 241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49272: BlockingRecv error: recv got EOF from 127.0.0.1:49272 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:38:40.756713 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15389
I20260620 11:38:40.762756 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15520
I20260620 11:38:40.767679 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15651
I20260620 11:38:40.772666 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15297
2026-06-20T11:38:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (566 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:40 dist-test-slave-szdr krb5kdc[15789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:40 dist-test-slave-szdr krb5kdc[15789](info): set up 2 sockets
Jun 20 11:38:40 dist-test-slave-szdr krb5kdc[15789](info): commencing operation
krb5kdc: starting...
W20260620 11:38:41.823553 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:41 dist-test-slave-szdr krb5kdc[15789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:38:41.949281 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39031
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:42.032639 15802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:42.032878 15802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:42.032903 15802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:42.034170 15802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:38:42.034212 15802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:42.034226 15802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:42.034240 15802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:38:42.034251 15802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:38:42.035734 15802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:39031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:42.035933 15802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:42.036121 15802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:42.041239 15808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:42.041282 15807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:42.041391 15802 server_base.cc:1034] running on GCE node
W20260620 11:38:42.041283 15810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:42.041821 15802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:42.042157 15802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:42.043318 15802 hybrid_clock.cc:648] HybridClock initialized: now 1781955522043295 us; error 48 us; skew 500 ppm
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955522, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:42.046027 15802 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:38:42.046562 15802 webserver.cc:460] Webserver started at http://127.5.122.190:39761/ using document root <none> and password file <none>
I20260620 11:38:42.046804 15802 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:42.046850 15802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:42.047005 15802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:42.048130 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "89e5ebb7761f436e832778aa67025d4e"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "b24f1800c70524fe1a72618775d71cbf"
server_key_iv: "d85d537d913a7bed635eb1df95945b79"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.048512 15802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "89e5ebb7761f436e832778aa67025d4e"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "b24f1800c70524fe1a72618775d71cbf"
server_key_iv: "d85d537d913a7bed635eb1df95945b79"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.049875 15802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:42.050675 15817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.050861 15802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:42.050976 15802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "89e5ebb7761f436e832778aa67025d4e"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "b24f1800c70524fe1a72618775d71cbf"
server_key_iv: "d85d537d913a7bed635eb1df95945b79"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.051064 15802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:42.064950 15802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:42.067585 15802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:42.067766 15802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:42.072008 15802 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:39031
I20260620 11:38:42.072044 15869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:39031 every 8 connection(s)
I20260620 11:38:42.072433 15802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:38:42.073499 15870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:42.074533 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15802
I20260620 11:38:42.074667 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:38:42.074864 5610 external_mini_cluster.cc:1428] Setting key 9865322aed2f0ed430584bad5ffd3695
I20260620 11:38:42.077124 15870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e: Bootstrap starting.
I20260620 11:38:42.077749 15870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:42.078106 15870 log.cc:824] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:42.078835 15870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e: No bootstrap required, opened a new log
I20260620 11:38:42.080619 15870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e5ebb7761f436e832778aa67025d4e" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39031 } }
I20260620 11:38:42.080786 15870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:42.080837 15870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89e5ebb7761f436e832778aa67025d4e, State: Initialized, Role: FOLLOWER
I20260620 11:38:42.080999 15870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e5ebb7761f436e832778aa67025d4e" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39031 } }
I20260620 11:38:42.081128 15870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:38:42.081163 15870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:38:42.081225 15870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 0 FOLLOWER]: Advancing to term 1
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:42.081996 15870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e5ebb7761f436e832778aa67025d4e" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39031 } }
I20260620 11:38:42.082130 15870 leader_election.cc:302] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89e5ebb7761f436e832778aa67025d4e; no voters:
I20260620 11:38:42.082309 15870 leader_election.cc:288] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:38:42.082346 15875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:42.082574 15870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:38:42.082581 15875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [term 1 LEADER]: Becoming Leader. State: Replica: 89e5ebb7761f436e832778aa67025d4e, State: Running, Role: LEADER
I20260620 11:38:42.082769 15875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e5ebb7761f436e832778aa67025d4e" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39031 } }
I20260620 11:38:42.083159 15876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89e5ebb7761f436e832778aa67025d4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e5ebb7761f436e832778aa67025d4e" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39031 } } }
I20260620 11:38:42.083411 15876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:42.083346 15877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89e5ebb7761f436e832778aa67025d4e. Latest consensus state: current_term: 1 leader_uuid: "89e5ebb7761f436e832778aa67025d4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89e5ebb7761f436e832778aa67025d4e" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39031 } } }
I20260620 11:38:42.083802 15877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e [sys.catalog]: This master's current role is: LEADER
I20260620 11:38:42.084044 15889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:38:42.084808 15889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:38:42.086586 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.075449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.5.122.190:39031)
0620 11:38:42.075640 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.075650 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.075688 (+ 38us) server_negotiation.cc:407] Connection header received
0620 11:38:42.076115 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.076123 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.076172 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.076290 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:42.077097 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.078170 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.079111 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.079534 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.079661 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.082320 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:42.082325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:42.082325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:42.082339 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:42.084835 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.085246 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.085249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.085250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.085360 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.085603 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.085605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.085605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.085916 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:42.086126 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.086216 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.086337 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260620 11:38:42.086843 15889 catalog_manager.cc:1347] Generated new cluster ID: 7cbb3dfc7fe34192aef6f33f8695f258
I20260620 11:38:42.086896 15889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:38:42.094782 15889 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:38:42.095499 15889 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:38:42.109824 15889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e: Generated new TSK 0
I20260620 11:38:42.110110 15889 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:38:42.148924 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39031
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:42.235873 15896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:42.236120 15896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:42.236145 15896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:42.237370 15896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:42.237417 15896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:42.237445 15896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:38:42.239014 15896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:42.239306 15896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:42.239552 15896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:42.244800 15902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:42.244827 15904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:42.244824 15901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:42.244961 15896 server_base.cc:1034] running on GCE node
I20260620 11:38:42.245249 15896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:42.245524 15896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:42.246694 15896 hybrid_clock.cc:648] HybridClock initialized: now 1781955522246667 us; error 49 us; skew 500 ppm
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955522, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:42.249300 15896 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:38:42.249780 15896 webserver.cc:460] Webserver started at http://127.5.122.129:33565/ using document root <none> and password file <none>
I20260620 11:38:42.250095 15896 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:42.250174 15896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:42.250342 15896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:42.251451 15896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "72c0153532af429d82b0878ec80706b2"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "85603600986f18cf0fc64162141cb19c"
server_key_iv: "e842be92d28c418d94d404317e629931"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.251910 15896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "72c0153532af429d82b0878ec80706b2"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "85603600986f18cf0fc64162141cb19c"
server_key_iv: "e842be92d28c418d94d404317e629931"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.253810 15896 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260620 11:38:42.254628 15911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.254912 15896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:42.255016 15896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "72c0153532af429d82b0878ec80706b2"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "85603600986f18cf0fc64162141cb19c"
server_key_iv: "e842be92d28c418d94d404317e629931"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.255110 15896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:42.268919 15896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:42.271262 15896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:42.271461 15896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:42.271744 15896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:42.272083 15896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:42.272140 15896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.272181 15896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:42.272207 15896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.278229 15896 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:33291
I20260620 11:38:42.278277 16024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:33291 every 8 connection(s)
I20260620 11:38:42.278625 15896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:38:42.284435 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 15896
I20260620 11:38:42.284549 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:38:42.284806 5610 external_mini_cluster.cc:1428] Setting key af4a1c2ab24532e525ec6b483e369bb6
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955522, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:42.289155 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.279356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:37555 (local address 127.5.122.190:39031)
0620 11:38:42.279433 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.279435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.279672 (+ 237us) server_negotiation.cc:407] Connection header received
0620 11:38:42.280163 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.280164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.280175 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.280230 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:42.281297 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.281814 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.282932 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.283222 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.283280 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.285879 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:42.285880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:42.285880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:42.285888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:42.287476 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.288037 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.288038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.288039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.288104 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.288365 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.288367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.288367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.288534 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:42.288606 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.288885 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.288999 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 11:38:42.289186 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.279351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39031 (local address 127.5.122.129:37555)
0620 11:38:42.279533 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.279575 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.280070 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.280360 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.280364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.280995 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:42.281205 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.281210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.281914 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.281915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.282818 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.282821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.283350 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.283352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.283479 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.283966 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:42.283975 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:42.285780 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:42.287564 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:42.287574 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:42.287576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:42.287937 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:42.288190 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:42.288190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:42.288190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:42.288301 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:42.288599 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:42.288603 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:42.288752 (+ 149us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.288840 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.288976 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260620 11:38:42.289886 16025 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39031
I20260620 11:38:42.290032 16025 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:42.290230 16025 heartbeater.cc:511] Master 127.5.122.190:39031 requested a full tablet report, sending...
I20260620 11:38:42.290814 15834 ts_manager.cc:194] Registered new tserver with Master: 72c0153532af429d82b0878ec80706b2 (127.5.122.129:33291)
I20260620 11:38:42.291465 15834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:37555
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:38:42.312896 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39031
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:42.396239 16030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:42.396512 16030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:42.396556 16030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:42.397821 16030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:42.397899 16030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:42.397948 16030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:38:42.399525 16030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:42.399837 16030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:42.400098 16030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:42.405391 16036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:42.405432 16038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:42.405386 16035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:42.405650 16030 server_base.cc:1034] running on GCE node
I20260620 11:38:42.405890 16030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:42.406229 16030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:42.407401 16030 hybrid_clock.cc:648] HybridClock initialized: now 1781955522407393 us; error 51 us; skew 500 ppm
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955522, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:42.410107 16030 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:38:42.410604 16030 webserver.cc:460] Webserver started at http://127.5.122.130:43993/ using document root <none> and password file <none>
I20260620 11:38:42.410863 16030 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:42.410933 16030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:42.411085 16030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:42.412168 16030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "8a1090763df94780969eada00c383841"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "3ee77ba495f6f74f73319dae30a15cb4"
server_key_iv: "3474a1078dafaf2ee7543229f9980663"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.412559 16030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "8a1090763df94780969eada00c383841"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "3ee77ba495f6f74f73319dae30a15cb4"
server_key_iv: "3474a1078dafaf2ee7543229f9980663"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.414052 16030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:38:42.414918 16045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.415100 16030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:38:42.415230 16030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "8a1090763df94780969eada00c383841"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "3ee77ba495f6f74f73319dae30a15cb4"
server_key_iv: "3474a1078dafaf2ee7543229f9980663"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.415311 16030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:42.441593 16030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:42.444065 16030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:42.444240 16030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:42.444482 16030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:42.444804 16030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:42.444860 16030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.444902 16030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:42.444931 16030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.450986 16030 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:44993
I20260620 11:38:42.451067 16158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:44993 every 8 connection(s)
I20260620 11:38:42.451412 16030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955522, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:42.458420 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 16030
I20260620 11:38:42.458534 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:38:42.458730 5610 external_mini_cluster.cc:1428] Setting key 14cd518ebfdcdd65591bb7841a8b769e
I20260620 11:38:42.460865 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.452119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:45989 (local address 127.5.122.190:39031)
0620 11:38:42.452161 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.452163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.452397 (+ 234us) server_negotiation.cc:407] Connection header received
0620 11:38:42.452882 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.452883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.452894 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.452946 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:42.453901 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.454452 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.455470 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.455681 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.455773 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.457888 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:42.457890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:42.457890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:42.457898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:42.459483 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.459884 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.459885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.459885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.459942 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.460219 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.460220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.460221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.460344 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:42.460419 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.460699 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.460776 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:38:42.461057 16161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.452108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39031 (local address 127.5.122.130:45989)
0620 11:38:42.452251 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.452287 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.452798 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.453001 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.453004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.453589 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:42.453783 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.453788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.454554 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.454555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.455361 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.455363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.455783 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.455785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.455913 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.456444 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:42.456452 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:42.457792 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:42.459568 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:42.459575 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:42.459577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:42.459806 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:42.460019 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:42.460020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:42.460021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:42.460131 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:42.460410 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:42.460416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:42.460559 (+ 143us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.460668 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.460824 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260620 11:38:42.461565 16159 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39031
I20260620 11:38:42.461676 16159 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:42.461848 16159 heartbeater.cc:511] Master 127.5.122.190:39031 requested a full tablet report, sending...
I20260620 11:38:42.462252 15834 ts_manager.cc:194] Registered new tserver with Master: 8a1090763df94780969eada00c383841 (127.5.122.130:44993)
I20260620 11:38:42.462632 15834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:45989
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:38:42.485891 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39031
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:38:42.569305 16164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:38:42.569540 16164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:38:42.569567 16164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:38:42.570859 16164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:38:42.570914 16164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:38:42.570943 16164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:38:42.572510 16164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:38:42.572794 16164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:38:42.573051 16164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:38:42.578460 16170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:38:42.578474 16172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:42.578526 16164 server_base.cc:1034] running on GCE node
W20260620 11:38:42.578460 16169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:38:42.578938 16164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:38:42.579223 16164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:38:42.580392 16164 hybrid_clock.cc:648] HybridClock initialized: now 1781955522580388 us; error 38 us; skew 500 ppm
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955522, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:38:42.583137 16164 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:38:42.583608 16164 webserver.cc:460] Webserver started at http://127.5.122.131:39583/ using document root <none> and password file <none>
I20260620 11:38:42.583935 16164 fs_manager.cc:359] Metadata directory not provided
I20260620 11:38:42.584022 16164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:38:42.584169 16164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:38:42.585286 16164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "ed022301867e4f5cad1a35810fda254e"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "2d1e71edce10f7f1f780b9aca9648dd7"
server_key_iv: "703fe54e3014bba3db8ba2fd6a28456e"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.585678 16164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "ed022301867e4f5cad1a35810fda254e"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "2d1e71edce10f7f1f780b9aca9648dd7"
server_key_iv: "703fe54e3014bba3db8ba2fd6a28456e"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.587113 16164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:38:42.587852 16179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.588030 16164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:38:42.588124 16164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "ed022301867e4f5cad1a35810fda254e"
format_stamp: "Formatted at 2026-06-20 11:38:42 on dist-test-slave-szdr"
server_key: "2d1e71edce10f7f1f780b9aca9648dd7"
server_key_iv: "703fe54e3014bba3db8ba2fd6a28456e"
server_key_version: "encryptionkey@0"
I20260620 11:38:42.588204 16164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:38:42.606880 16164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:38:42.608891 16164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:38:42.609068 16164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:38:42.609320 16164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:38:42.609750 16164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:38:42.609810 16164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.609844 16164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:38:42.609861 16164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:38:42.616058 16164 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:37929
I20260620 11:38:42.616140 16292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:37929 every 8 connection(s)
I20260620 11:38:42.616497 16164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:38:42.621310 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 16164
I20260620 11:38:42.621434 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:38:42.621647 5610 external_mini_cluster.cc:1428] Setting key 07345bc7e43adddbddaa9386834ea7fd
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[15789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955522, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:38:42.626855 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.617325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:43689 (local address 127.5.122.190:39031)
0620 11:38:42.617414 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.617416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.617648 (+ 232us) server_negotiation.cc:407] Connection header received
0620 11:38:42.618190 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.618191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.618202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.618319 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:42.619321 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.619871 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.620903 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.621096 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.621194 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.623806 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:42.623808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:42.623808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:42.623817 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:42.625180 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.625684 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.625685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.625686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.625758 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.626075 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.626076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.626077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.626210 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:42.626282 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.626594 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.626736 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 11:38:42.626991 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.617290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39031 (local address 127.5.122.131:43689)
0620 11:38:42.617461 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.617506 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.618094 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.618396 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.618400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.619006 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:38:42.619215 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.619220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.619959 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.619960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.620808 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.620811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.621229 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.621231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.621377 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.622095 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:38:42.622106 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:38:42.623693 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:38:42.625308 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:42.625317 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:42.625319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:42.625592 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:42.625836 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:38:42.625837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:38:42.625837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:38:42.626006 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:38:42.626284 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:38:42.626291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:38:42.626453 (+ 162us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.626560 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.626746 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260620 11:38:42.627554 16293 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39031
I20260620 11:38:42.627663 16293 heartbeater.cc:464] Registering TS with master...
I20260620 11:38:42.627826 16293 heartbeater.cc:511] Master 127.5.122.190:39031 requested a full tablet report, sending...
I20260620 11:38:42.628216 15834 ts_manager.cc:194] Registered new tserver with Master: ed022301867e4f5cad1a35810fda254e (127.5.122.131:37929)
I20260620 11:38:42.628608 15834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:43689
I20260620 11:38:42.633477 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:38:42.641681 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.635054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.5.122.190:39031)
0620 11:38:42.635150 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.635152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.635254 (+ 102us) server_negotiation.cc:407] Connection header received
0620 11:38:42.635351 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.635352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.635364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.635423 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:38:42.636041 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.636557 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.637307 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.637522 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.637610 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.638996 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:38:42.638997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:38:42.638998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:38:42.639006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:38:42.640402 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.640773 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.640774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.640775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.640837 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:38:42.641050 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:38:42.641051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:38:42.641051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:38:42.641223 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:38:42.641338 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.641480 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.641572 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 11:38:42.643386 15834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:38:42.645156 15834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:38:42.652046 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.647774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:37929 (local address 127.0.0.1:45318)
0620 11:38:42.647942 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.647985 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.648052 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.648376 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.648380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.648382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:42.648465 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.648467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.649465 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.649466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.650666 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.650669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.651697 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.651698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.651781 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.651796 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.651877 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.651936 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":58,"threads_started":1}
I20260620 11:38:42.652048 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.647890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.5.122.131:37929)
0620 11:38:42.648090 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.648095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.648112 (+ 17us) server_negotiation.cc:407] Connection header received
0620 11:38:42.648158 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.648160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.648178 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.648255 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:42.648556 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.649359 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.650764 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.651607 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.651686 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.651748 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.651906 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.651950 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
I20260620 11:38:42.652426 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.647974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:44993 (local address 127.0.0.1:53966)
0620 11:38:42.648243 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.648254 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.648299 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.648635 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.648636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.648638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:42.648679 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.648681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.650073 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.650075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.651080 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.651082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.652112 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.652113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.652189 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.652193 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.652242 (+ 49us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.652286 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":60,"thread_start_us":76,"threads_started":1}
I20260620 11:38:42.652429 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.647774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:33291 (local address 127.0.0.1:44036)
0620 11:38:42.648094 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.648105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.648181 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.648667 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.648668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.648671 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:42.648728 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.648730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.650093 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.650095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.651116 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.651119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.652147 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.652149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.652237 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.652242 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.652275 (+ 33us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.652316 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":6,"thread_start_us":93,"threads_started":1}
I20260620 11:38:42.652530 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.648204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53966 (local address 127.5.122.130:44993)
0620 11:38:42.648410 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.648415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.648425 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:42.648461 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.648463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.648480 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.648554 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:42.648746 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.649916 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.651177 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.651963 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.652026 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.652076 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.652387 (+ 311us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.652436 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":102,"threads_started":1}
I20260620 11:38:42.653088 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.647890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.5.122.129:33291)
0620 11:38:42.648494 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.648499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.648509 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:38:42.648547 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.648549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.648566 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.648807 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:42.648888 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.649941 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.651282 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.652040 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.652863 (+ 823us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.652933 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.652958 (+ 25us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.653004 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":89,"threads_started":1}
I20260620 11:38:42.653275 16093 tablet_service.cc:1467] Processing CreateTablet for tablet 6dea7e9613104b47aedab0fa0f9cde66 (DEFAULT_TABLE table=test-table [id=d893e709d94c40b0aea470ff54f986cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:42.653792 16093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dea7e9613104b47aedab0fa0f9cde66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:42.653836 15959 tablet_service.cc:1467] Processing CreateTablet for tablet 6dea7e9613104b47aedab0fa0f9cde66 (DEFAULT_TABLE table=test-table [id=d893e709d94c40b0aea470ff54f986cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:42.654217 15959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dea7e9613104b47aedab0fa0f9cde66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:42.654469 16227 tablet_service.cc:1467] Processing CreateTablet for tablet 6dea7e9613104b47aedab0fa0f9cde66 (DEFAULT_TABLE table=test-table [id=d893e709d94c40b0aea470ff54f986cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:38:42.654809 16227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dea7e9613104b47aedab0fa0f9cde66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:38:42.656951 16310 tablet_bootstrap.cc:492] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2: Bootstrap starting.
I20260620 11:38:42.657430 16311 tablet_bootstrap.cc:492] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Bootstrap starting.
I20260620 11:38:42.658052 16310 tablet_bootstrap.cc:654] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:42.658145 16311 tablet_bootstrap.cc:654] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:42.658711 16312 tablet_bootstrap.cc:492] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Bootstrap starting.
I20260620 11:38:42.658784 16311 log.cc:824] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:42.658784 16310 log.cc:824] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:42.659554 16312 tablet_bootstrap.cc:654] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Neither blocks nor log segments found. Creating new log.
I20260620 11:38:42.659649 16310 tablet_bootstrap.cc:492] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2: No bootstrap required, opened a new log
I20260620 11:38:42.659730 16310 ts_tablet_manager.cc:1397] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260620 11:38:42.659874 16311 tablet_bootstrap.cc:492] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: No bootstrap required, opened a new log
I20260620 11:38:42.659940 16311 ts_tablet_manager.cc:1397] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:38:42.660163 16312 log.cc:824] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Log is configured to *not* fsync() on all Append() calls
I20260620 11:38:42.661793 16310 raft_consensus.cc:348] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.662019 16310 raft_consensus.cc:374] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:42.661895 16311 raft_consensus.cc:348] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.662061 16310 raft_consensus.cc:729] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c0153532af429d82b0878ec80706b2, State: Initialized, Role: FOLLOWER
I20260620 11:38:42.662070 16311 raft_consensus.cc:374] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:42.662118 16311 raft_consensus.cc:729] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1090763df94780969eada00c383841, State: Initialized, Role: FOLLOWER
I20260620 11:38:42.662225 16311 consensus_queue.cc:260] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.662220 16310 consensus_queue.cc:260] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.662521 16310 ts_tablet_manager.cc:1428] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 11:38:42.662662 16159 heartbeater.cc:503] Master 127.5.122.190:39031 was elected leader, sending a full tablet report...
I20260620 11:38:42.662673 16025 heartbeater.cc:503] Master 127.5.122.190:39031 was elected leader, sending a full tablet report...
I20260620 11:38:42.662906 16312 tablet_bootstrap.cc:492] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: No bootstrap required, opened a new log
I20260620 11:38:42.662949 16311 ts_tablet_manager.cc:1428] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:38:42.662982 16312 ts_tablet_manager.cc:1397] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 11:38:42.664943 16312 raft_consensus.cc:348] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.665118 16312 raft_consensus.cc:374] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:38:42.665169 16312 raft_consensus.cc:729] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed022301867e4f5cad1a35810fda254e, State: Initialized, Role: FOLLOWER
I20260620 11:38:42.665294 16312 consensus_queue.cc:260] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.665611 16312 ts_tablet_manager.cc:1428] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:38:42.665711 16293 heartbeater.cc:503] Master 127.5.122.190:39031 was elected leader, sending a full tablet report...
I20260620 11:38:42.679533 16316 raft_consensus.cc:482] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:38:42.679661 16316 raft_consensus.cc:504] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.680140 16316 leader_election.cc:288] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed022301867e4f5cad1a35810fda254e (127.5.122.131:37929), 8a1090763df94780969eada00c383841 (127.5.122.130:44993)
I20260620 11:38:42.683573 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.680424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:44993 (local address 127.5.122.129:56821)
0620 11:38:42.680625 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.680637 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.680687 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.680866 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.680867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.680869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:42.680955 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.680957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.681596 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.681597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.682456 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.682458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.683224 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.683225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.683298 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.683302 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.683384 (+ 82us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.683434 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":124,"threads_started":1}
I20260620 11:38:42.683588 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.680588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:56821 (local address 127.5.122.130:44993)
0620 11:38:42.680660 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.680662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.680673 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:38:42.680766 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.680767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.680778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.680839 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:42.681026 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.681507 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.682587 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.683097 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.683175 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.683186 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.683405 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.683491 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:38:42.683656 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.680445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:53251 (local address 127.5.122.131:37929)
0620 11:38:42.680529 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.680532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.680541 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:38:42.680570 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.680571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.680582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.680655 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:38:42.680892 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.681440 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.682543 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.683074 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.683187 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.683196 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.683526 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.683570 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 11:38:42.683930 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.680314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:37929 (local address 127.5.122.129:53251)
0620 11:38:42.680352 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.680367 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:38:42.680410 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:38:42.680709 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:38:42.680710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:38:42.680712 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:38:42.680813 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.680816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.681521 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.681522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.682450 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:38:42.682452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.683245 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.683246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:38:42.683354 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.683357 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:38:42.683404 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:38:42.683451 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":31}
I20260620 11:38:42.684121 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dea7e9613104b47aedab0fa0f9cde66" candidate_uuid: "72c0153532af429d82b0878ec80706b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1090763df94780969eada00c383841" is_pre_election: true
I20260620 11:38:42.684329 16113 raft_consensus.cc:2454] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72c0153532af429d82b0878ec80706b2 in term 0.
I20260620 11:38:42.684623 15915 leader_election.cc:302] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72c0153532af429d82b0878ec80706b2, 8a1090763df94780969eada00c383841; no voters:
I20260620 11:38:42.684700 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dea7e9613104b47aedab0fa0f9cde66" candidate_uuid: "72c0153532af429d82b0878ec80706b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed022301867e4f5cad1a35810fda254e" is_pre_election: true
I20260620 11:38:42.684760 16316 raft_consensus.cc:2781] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:38:42.684814 16316 raft_consensus.cc:482] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:38:42.684836 16316 raft_consensus.cc:3037] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:42.684875 16247 raft_consensus.cc:2454] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72c0153532af429d82b0878ec80706b2 in term 0.
I20260620 11:38:42.685716 16316 raft_consensus.cc:504] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.685869 16316 leader_election.cc:288] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [CANDIDATE]: Term 1 election: Requested vote from peers ed022301867e4f5cad1a35810fda254e (127.5.122.131:37929), 8a1090763df94780969eada00c383841 (127.5.122.130:44993)
I20260620 11:38:42.686050 16247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dea7e9613104b47aedab0fa0f9cde66" candidate_uuid: "72c0153532af429d82b0878ec80706b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed022301867e4f5cad1a35810fda254e"
I20260620 11:38:42.686146 16247 raft_consensus.cc:3037] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:42.686152 16113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dea7e9613104b47aedab0fa0f9cde66" candidate_uuid: "72c0153532af429d82b0878ec80706b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a1090763df94780969eada00c383841"
I20260620 11:38:42.686237 16113 raft_consensus.cc:3037] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:38:42.686949 16247 raft_consensus.cc:2454] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72c0153532af429d82b0878ec80706b2 in term 1.
I20260620 11:38:42.687090 15915 leader_election.cc:302] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72c0153532af429d82b0878ec80706b2, ed022301867e4f5cad1a35810fda254e; no voters:
I20260620 11:38:42.687099 16113 raft_consensus.cc:2454] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72c0153532af429d82b0878ec80706b2 in term 1.
I20260620 11:38:42.687191 16316 raft_consensus.cc:2781] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:38:42.687386 16316 raft_consensus.cc:686] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [term 1 LEADER]: Becoming Leader. State: Replica: 72c0153532af429d82b0878ec80706b2, State: Running, Role: LEADER
I20260620 11:38:42.687492 16316 consensus_queue.cc:237] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } }
I20260620 11:38:42.688500 15833 catalog_manager.cc:5495] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72c0153532af429d82b0878ec80706b2 (127.5.122.129). New cstate: current_term: 1 leader_uuid: "72c0153532af429d82b0878ec80706b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72c0153532af429d82b0878ec80706b2" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 } health_report { overall_health: UNKNOWN } } }
W20260620 11:38:42.702026 16160 tablet.cc:2369] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:38:42.702749 16306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.700266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.5.122.129:33291)
0620 11:38:42.700349 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 11:38:42.700351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:38:42.700359 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:38:42.700408 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:38:42.700409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:38:42.700428 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:38:42.700494 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:38:42.700636 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.701110 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.701836 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:38:42.702103 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:38:42.702184 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:38:42.702372 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:38:42.702495 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:38:42.702596 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 11:38:42.702615 (+ 19us) server_negotiation.cc:299] Negotiation successful
0620 11:38:42.702639 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 11:38:42.704835 16247 raft_consensus.cc:1261] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 1 FOLLOWER]: Refusing update from remote peer 72c0153532af429d82b0878ec80706b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:42.704841 16113 raft_consensus.cc:1261] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 1 FOLLOWER]: Refusing update from remote peer 72c0153532af429d82b0878ec80706b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:38:42.705283 16320 consensus_queue.cc:1035] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed022301867e4f5cad1a35810fda254e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:42.705430 16316 consensus_queue.cc:1035] T 6dea7e9613104b47aedab0fa0f9cde66 P 72c0153532af429d82b0878ec80706b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a1090763df94780969eada00c383841" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:38:42.707933 16323 mvcc.cc:205] Tried to move back new op lower bound from 7298889820995063808 to 7298889820928385024. Current Snapshot: MvccSnapshot[applied={T|T < 7298889820995063808}]
I20260620 11:38:42.709059 16324 mvcc.cc:205] Tried to move back new op lower bound from 7298889820995063808 to 7298889820928385024. Current Snapshot: MvccSnapshot[applied={T|T < 7298889820995063808}]
I20260620 11:38:42.710448 15834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:42.710546 15834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:38:42.711333 15834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89e5ebb7761f436e832778aa67025d4e: Sending DeleteTablet for 3 replicas of tablet 6dea7e9613104b47aedab0fa0f9cde66
I20260620 11:38:42.711689 15959 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dea7e9613104b47aedab0fa0f9cde66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:42 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:44036
I20260620 11:38:42.711709 16227 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dea7e9613104b47aedab0fa0f9cde66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:42 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:45318
I20260620 11:38:42.711920 16093 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dea7e9613104b47aedab0fa0f9cde66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:38:42 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:53966
I20260620 11:38:42.712144 16334 tablet_replica.cc:330] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: stopping tablet replica
I20260620 11:38:42.712234 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15896
I20260620 11:38:42.712283 16334 raft_consensus.cc:2229] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:42.713943 16334 raft_consensus.cc:2258] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:42.714263 16332 tablet_replica.cc:330] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: stopping tablet replica
I20260620 11:38:42.714376 16332 raft_consensus.cc:2229] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:38:42.714478 16332 raft_consensus.cc:2258] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:38:42.714607 16334 ts_tablet_manager.cc:1905] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:42.714915 16332 ts_tablet_manager.cc:1905] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:38:42.716527 16332 ts_tablet_manager.cc:1918] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:42.716632 16332 log.cc:1196] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/6dea7e9613104b47aedab0fa0f9cde66
I20260620 11:38:42.716681 16334 ts_tablet_manager.cc:1918] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:38:42.716763 16334 log.cc:1196] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/6dea7e9613104b47aedab0fa0f9cde66
I20260620 11:38:42.716928 16332 ts_tablet_manager.cc:1939] T 6dea7e9613104b47aedab0fa0f9cde66 P ed022301867e4f5cad1a35810fda254e: Deleting consensus metadata
I20260620 11:38:42.717051 16334 ts_tablet_manager.cc:1939] T 6dea7e9613104b47aedab0fa0f9cde66 P 8a1090763df94780969eada00c383841: Deleting consensus metadata
I20260620 11:38:42.717453 15821 catalog_manager.cc:4841] TS 8a1090763df94780969eada00c383841 (127.5.122.130:44993): tablet 6dea7e9613104b47aedab0fa0f9cde66 (table test-table [id=d893e709d94c40b0aea470ff54f986cb]) successfully deleted
I20260620 11:38:42.717585 15821 catalog_manager.cc:4841] TS ed022301867e4f5cad1a35810fda254e (127.5.122.131:37929): tablet 6dea7e9613104b47aedab0fa0f9cde66 (table test-table [id=d893e709d94c40b0aea470ff54f986cb]) successfully deleted
W20260620 11:38:42.718793 15820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:33291 (error 108)
I20260620 11:38:42.718993 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 16030
I20260620 11:38:42.719635 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:38:42.719174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:33291 (local address 127.0.0.1:44052)
0620 11:38:42.719539 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0620 11:38:42.719591 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:33291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337}
W20260620 11:38:42.719848 15820 catalog_manager.cc:4584] TS 72c0153532af429d82b0878ec80706b2 (127.5.122.129:33291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dea7e9613104b47aedab0fa0f9cde66: Network error: Client connection negotiation failed: client connection to 127.5.122.129:33291: connect: Connection refused (error 111)
I20260620 11:38:42.724123 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 16164
I20260620 11:38:42.728895 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 15802
2026-06-20T11:38:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1956 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[16338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[16338](info): set up 2 sockets
Jun 20 11:38:42 dist-test-slave-szdr krb5kdc[16338](info): commencing operation
krb5kdc: starting...
W20260620 11:38:43.763123 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:38:43 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:38:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:38:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/rangeradmin_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/rangeradmin_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/rangerlookup_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/rangerlookup_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
I20260620 11:38:43.932854 5610 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJIe33o/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/postgres -l logfile start
2026-06-20 11:38:46.593 UTC [16368] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 11:38:46.593 UTC [16368] LOG: listening on IPv4 address "127.5.122.148", port 52523
2026-06-20 11:38:46.597 UTC [16368] LOG: listening on Unix socket "/tmp/.s.PGSQL.52523"
2026-06-20 11:38:46.633 UTC [16371] LOG: database system was shut down at 2026-06-20 11:38:45 UTC
2026-06-20 11:38:46.637 UTC [16368] LOG: database system is ready to accept connections
I20260620 11:38:47.599083 5610 mini_postgres.cc:96] Postgres bound to 52523
2026-06-20 11:38:47.605 UTC [16379] FATAL: database "slave" does not exist
127.5.122.148:52523 - accepting connections
I20260620 11:38:47.606200 5610 mini_ranger.cc:161] Starting Ranger...
I20260620 11:38:47.622344 5610 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 11:38:48.126438 5610 mini_ranger.cc:87] Created ranger Postgres database
I20260620 11:38:48.126654 5610 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-admin
2026-06-20 11:38:48,410 [I] DB FLAVOR :POSTGRES
2026-06-20 11:38:48,410 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 11:38:48,410 [I] Checking connection..
2026-06-20 11:38:48,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 11:38:48,647 [I] Checking connection passed.
2026-06-20 11:38:48,647 [I] --------- Verifying version history table ---------
2026-06-20 11:38:48,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 11:38:48,899 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 11:38:48,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 11:38:49,147 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 11:38:49,148 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 11:38:49,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 11:38:49.420 UTC [16443] WARNING: there is no transaction in progress
2026-06-20 11:38:49,426 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 11:38:49,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 11:38:49,695 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 11:38:49,696 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 11:38:49,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 11:38:49,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 11:38:50,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-szdr.c.gcp-upstream.internal','N') ;"
2026-06-20 11:38:50,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 11:38:50,720 [I] Table x_portal_user does not exist in database ranger
2026-06-20 11:38:50,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 11:38:50,979 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 11:38:50,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 11:38:51,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 11:38:51,511 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 11:38:51,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 11:38:52.552 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.570 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.588 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.605 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.623 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.644 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.706 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.713 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.727 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.739 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.752 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.768 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.779 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.787 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.795 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:52.803 UTC [16551] WARNING: there is no transaction in progress
2026-06-20 11:38:53,528 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 11:38:53,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 11:38:53,770 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 11:38:53,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 11:38:54,025 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 11:38:54,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 11:38:54,268 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 11:38:54,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 11:38:54,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 11:38:54,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-szdr.c.gcp-upstream.internal';"
2026-06-20 11:38:55,025 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 11:38:55,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 11:38:55,265 [I] DB_PATCHES have already been applied
I20260620 11:38:55.271351 5610 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJIe33o/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 11:38:55.271430 5610 mini_ranger.cc:193] Using host: 127.5.122.148
I20260620 11:38:55.274626 5610 mini_ranger.cc:227] Ranger admin URL: http://127.5.122.148:48139
Jun 20, 2026 11:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:38:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:38:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:38:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 11:38:56.400017 5610 mini_ranger.cc:160] Time spent starting Ranger: real 8.794s user 0.003s sys 0.006s
Jun 20, 2026 11:39:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:39:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 11:39:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:39:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:39:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 11:39:19.631417 5610 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/rangerkms_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/rangerkms_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.122.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 11:39:19 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955559, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 11:39:19.722222 5610 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 11:39:19.732470 5610 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 11:39:20.689334 5610 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 11:39:20.689432 5610 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms
I20260620 11:39:20.689452 5610 mini_ranger_kms.cc:227] Using postgres at 127.5.122.148:52523
2026-06-20 11:39:20,865 [I] DB FLAVOR :POSTGRES
2026-06-20 11:39:20,865 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 11:39:20,865 [I] Checking connection
2026-06-20 11:39:20,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 11:39:21,115 [I] connection success
2026-06-20 11:39:21,115 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 11:39:21,116 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 11:39:21,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 11:39:21,359 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 11:39:21,359 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 11:39:21,360 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 11:39:21,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:52523/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 11:39:21,625 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 11:39:22.169564 5610 mini_ranger_kms.cc:310] Created kms service
I20260620 11:39:22.380218 5610 mini_ranger_kms.cc:326] Created kudu user
I20260620 11:39:22.428742 5610 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 11:39:22 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955559, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:39:22.864408 5610 mini_ranger_kms.cc:384] Added ranger policy
I20260620 11:39:22.864532 5610 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 11:39:22.864555 5610 mini_ranger_kms.cc:240] Using host: 127.5.122.148
I20260620 11:39:22.868515 5610 mini_ranger_kms.cc:276] Ranger KMS PID: 16732
I20260620 11:39:22.868674 5610 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.122.148:33065
Jun 20, 2026 11:39:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:39:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:39:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:39:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:39:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:39:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 11:39:25.174266 5610 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.452s user 0.007s sys 0.151s
I20260620 11:39:25.174383 5610 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 11:39:25.174459 5610 mini_ranger_kms.cc:401] 127.5.122.148:33065/kms/v1/keys
Jun 20 11:39:25 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781955565, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 11:39:25 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955565, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.148@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
Jun 20 11:39:26 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955565, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.148@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
Jun 20 11:39:28 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:39:28.788851 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:45889
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:45889
--ranger_config_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:33065/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:39:28.874366 16782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:39:28.874619 16782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:39:28.874657 16782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:39:28.875953 16782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:39:28.876030 16782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:39:28.876055 16782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:39:28.876087 16782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:39:28.876112 16782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:39:28.877789 16782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.122.148:33065/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:45889
--ranger_config_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:45889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:39:28.878119 16782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:39:28.878371 16782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:39:28.884063 16782 server_base.cc:1034] running on GCE node
W20260620 11:39:28.884229 16787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:28.884233 16790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:28.884245 16788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:39:28.884768 16782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:39:28.885090 16782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:39:28.886261 16782 hybrid_clock.cc:648] HybridClock initialized: now 1781955568886233 us; error 52 us; skew 500 ppm
Jun 20 11:39:28 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955568, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:39:28.889035 16782 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:39:28.889500 16782 webserver.cc:460] Webserver started at http://127.5.122.190:39941/ using document root <none> and password file <none>
I20260620 11:39:28.889739 16782 fs_manager.cc:359] Metadata directory not provided
I20260620 11:39:28.889791 16782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:39:28.889930 16782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:39:28 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955568, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:39:28.947229 16782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "caf1a1bc82f84fb6bccaf3896341112c"
format_stamp: "Formatted at 2026-06-20 11:39:28 on dist-test-slave-szdr"
server_key: "3b45d788015647030459122f8c715be1"
server_key_iv: "f93431e2fb0a67f6051c08bf7e390cde"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:28.947659 16782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "caf1a1bc82f84fb6bccaf3896341112c"
format_stamp: "Formatted at 2026-06-20 11:39:28 on dist-test-slave-szdr"
server_key: "3b45d788015647030459122f8c715be1"
server_key_iv: "f93431e2fb0a67f6051c08bf7e390cde"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:28.949224 16782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:39:28.961831 16797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:28.962049 16782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:39:28.962162 16782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "caf1a1bc82f84fb6bccaf3896341112c"
format_stamp: "Formatted at 2026-06-20 11:39:28 on dist-test-slave-szdr"
server_key: "3b45d788015647030459122f8c715be1"
server_key_iv: "f93431e2fb0a67f6051c08bf7e390cde"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:28.962284 16782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:39:28.984426 16782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:39:28.986716 16782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:39:28.986920 16782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:39:28.991818 16782 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:45889
I20260620 11:39:28.991838 16849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:45889 every 8 connection(s)
I20260620 11:39:28.992265 16782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:39:28.992946 16850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:39:28.995415 16850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c: Bootstrap starting.
I20260620 11:39:28.995545 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 16782
I20260620 11:39:28.995661 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:39:28.995846 5610 external_mini_cluster.cc:1428] Setting key 116ffda22b7c6d292e733805a65b71cb
I20260620 11:39:28.996167 16850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c: Neither blocks nor log segments found. Creating new log.
I20260620 11:39:28.996560 16850 log.cc:824] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c: Log is configured to *not* fsync() on all Append() calls
I20260620 11:39:28.997303 16850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c: No bootstrap required, opened a new log
I20260620 11:39:28.999630 16850 raft_consensus.cc:348] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf1a1bc82f84fb6bccaf3896341112c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45889 } }
I20260620 11:39:28.999806 16850 raft_consensus.cc:374] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:39:28.999852 16850 raft_consensus.cc:729] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf1a1bc82f84fb6bccaf3896341112c, State: Initialized, Role: FOLLOWER
I20260620 11:39:29.000005 16850 consensus_queue.cc:260] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf1a1bc82f84fb6bccaf3896341112c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45889 } }
I20260620 11:39:29.000097 16850 raft_consensus.cc:388] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:39:29.000130 16850 raft_consensus.cc:482] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:39:29.000176 16850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:39:29.000928 16850 raft_consensus.cc:504] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf1a1bc82f84fb6bccaf3896341112c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45889 } }
I20260620 11:39:29.001044 16850 leader_election.cc:302] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caf1a1bc82f84fb6bccaf3896341112c; no voters:
I20260620 11:39:29.001261 16850 leader_election.cc:288] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:39:29.001286 16855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:39:29.001406 16855 raft_consensus.cc:686] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [term 1 LEADER]: Becoming Leader. State: Replica: caf1a1bc82f84fb6bccaf3896341112c, State: Running, Role: LEADER
I20260620 11:39:29.001500 16855 consensus_queue.cc:237] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf1a1bc82f84fb6bccaf3896341112c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45889 } }
I20260620 11:39:29.001571 16850 sys_catalog.cc:564] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:39:29.001852 16855 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caf1a1bc82f84fb6bccaf3896341112c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf1a1bc82f84fb6bccaf3896341112c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45889 } } }
I20260620 11:39:29.001920 16855 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [sys.catalog]: This master's current role is: LEADER
I20260620 11:39:29.002086 16855 sys_catalog.cc:455] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [sys.catalog]: SysCatalogTable state changed. Reason: New leader caf1a1bc82f84fb6bccaf3896341112c. Latest consensus state: current_term: 1 leader_uuid: "caf1a1bc82f84fb6bccaf3896341112c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caf1a1bc82f84fb6bccaf3896341112c" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45889 } } }
I20260620 11:39:29.002158 16855 sys_catalog.cc:458] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c [sys.catalog]: This master's current role is: LEADER
I20260620 11:39:29.003118 16850 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 11:39:29.003301 16860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 20 11:39:28 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:39:29.006820 16860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:39:29.010926 16860 catalog_manager.cc:1347] Generated new cluster ID: aaa1f123766640ab879af6c80f16c1a8
I20260620 11:39:29.010978 16860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:39:29.013499 16853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:28.996427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42612 (local address 127.5.122.190:45889)
0620 11:39:28.996622 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:28.996631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:28.996666 (+ 35us) server_negotiation.cc:407] Connection header received
0620 11:39:28.997351 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:28.997362 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:28.997431 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:28.997596 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:39:28.998412 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:28.999837 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:29.001085 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:29.002228 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:29.002350 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:29.005758 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:39:29.005808 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:39:29.005809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:39:29.005912 (+ 103us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:39:29.009857 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:29.011241 (+ 1384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:29.011259 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:29.011259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:29.011377 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:29.011646 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:29.011647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:29.011647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:29.012115 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:39:29.012361 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:29.012831 (+ 470us) server_negotiation.cc:299] Negotiation successful
0620 11:39:29.013143 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
I20260620 11:39:29.024330 16860 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:39:29.025116 16860 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:39:29.035158 16860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c: Generated new TSK 0
I20260620 11:39:29.035476 16860 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:39:29.785591 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:33065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:45889
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:39:29.904347 16895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:39:29.904639 16895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:39:29.904688 16895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:39:29.906322 16895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:39:29.906409 16895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:39:29.906464 16895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:39:29.908816 16895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.122.148:33065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:45889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:39:29.909181 16895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:39:29.909468 16895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:39:29.916322 16903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:29.916306 16900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:29.916273 16901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:39:29.917230 16895 server_base.cc:1034] running on GCE node
I20260620 11:39:29.917491 16895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:39:29.917996 16895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:39:29.919214 16895 hybrid_clock.cc:648] HybridClock initialized: now 1781955569919187 us; error 49 us; skew 500 ppm
Jun 20 11:39:29 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955569, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:39:29.922577 16895 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:39:29.923237 16895 webserver.cc:460] Webserver started at http://127.5.122.129:42245/ using document root <none> and password file <none>
I20260620 11:39:29.923555 16895 fs_manager.cc:359] Metadata directory not provided
I20260620 11:39:29.923647 16895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:39:29.923811 16895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:39:29 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955569, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:39:29.941525 16895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "5566e12633364bb7b1ac45e4543790c6"
format_stamp: "Formatted at 2026-06-20 11:39:29 on dist-test-slave-szdr"
server_key: "95f4d211b000b4d999913cc60aae7826"
server_key_iv: "d6561aaadb4d72c18d52c36197335ef1"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:29.942093 16895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "5566e12633364bb7b1ac45e4543790c6"
format_stamp: "Formatted at 2026-06-20 11:39:29 on dist-test-slave-szdr"
server_key: "95f4d211b000b4d999913cc60aae7826"
server_key_iv: "d6561aaadb4d72c18d52c36197335ef1"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:29.946519 16895 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260620 11:39:29.965454 16912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:29.965698 16895 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 11:39:29.965806 16895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "5566e12633364bb7b1ac45e4543790c6"
format_stamp: "Formatted at 2026-06-20 11:39:29 on dist-test-slave-szdr"
server_key: "95f4d211b000b4d999913cc60aae7826"
server_key_iv: "d6561aaadb4d72c18d52c36197335ef1"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:29.965932 16895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:39:29.979725 16895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:39:29.982539 16895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:39:29.982717 16895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:39:29.983016 16895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:39:29.983438 16895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:39:29.983507 16895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:29.983546 16895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:39:29.983572 16895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:29.991377 16895 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:39903
I20260620 11:39:29.992005 16895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:39:29.992604 17025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:39903 every 8 connection(s)
I20260620 11:39:29.995831 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 16895
I20260620 11:39:29.995950 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:39:29.996153 5610 external_mini_cluster.cc:1428] Setting key bfdef83b9a2a9ef3b3bb16ec2084520c
Jun 20 11:39:29 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955569, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:39:30.005882 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:29.993415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:45889 (local address 127.5.122.129:38069)
0620 11:39:29.994116 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:29.994167 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:29.994794 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:29.995252 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:29.995258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:29.995535 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:39:29.995843 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:29.995850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:29.996870 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:29.996872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:29.998247 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:29.998250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:29.998869 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:29.998870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:29.999019 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:29.999699 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:39:29.999710 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:39:30.001449 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:39:30.003780 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:39:30.003797 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:39:30.003802 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:39:30.004144 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:39:30.004394 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:39:30.004395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:39:30.004395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:39:30.004556 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:39:30.004909 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:39:30.004916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:39:30.005100 (+ 184us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.005221 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.005564 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":83,"threads_started":1}
I20260620 11:39:30.006421 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:29.993757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:38069 (local address 127.5.122.190:45889)
0620 11:39:29.994990 (+ 1233us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:29.994992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:29.995000 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:39:29.995024 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:29.995025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:29.995037 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:29.995105 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:39:29.995967 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:29.996750 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:29.998355 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:29.998666 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:29.998751 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.001584 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:39:30.001586 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:39:30.001586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:39:30.001595 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:39:30.003645 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.004241 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.004242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.004242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.004318 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.004636 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.004637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.004638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.004811 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:39:30.004879 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.006218 (+ 1339us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.006325 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":79,"threads_started":1}
I20260620 11:39:30.006807 17026 heartbeater.cc:347] Connected to a master server at 127.5.122.190:45889
I20260620 11:39:30.006917 17026 heartbeater.cc:464] Registering TS with master...
I20260620 11:39:30.007148 17026 heartbeater.cc:511] Master 127.5.122.190:45889 requested a full tablet report, sending...
I20260620 11:39:30.007772 16814 ts_manager.cc:194] Registered new tserver with Master: 5566e12633364bb7b1ac45e4543790c6 (127.5.122.129:39903)
I20260620 11:39:30.008635 16814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:38069
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:39:30.049042 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:33065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:45889
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 11:39:30.209940 17032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:39:30.210294 17032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:39:30.210338 17032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:39:30.212257 17032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:39:30.212339 17032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:39:30.212389 17032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:39:30.214955 17032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.122.148:33065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:45889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:39:30.215293 17032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:39:30.215582 17032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:39:30.222645 17040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:30.223008 17042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:30.223881 17039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:39:30.224151 17032 server_base.cc:1034] running on GCE node
I20260620 11:39:30.224398 17032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:39:30.224749 17032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:39:30.225951 17032 hybrid_clock.cc:648] HybridClock initialized: now 1781955570225926 us; error 36 us; skew 500 ppm
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:39:30.229840 17032 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:39:30.230450 17032 webserver.cc:460] Webserver started at http://127.5.122.130:45657/ using document root <none> and password file <none>
I20260620 11:39:30.230769 17032 fs_manager.cc:359] Metadata directory not provided
I20260620 11:39:30.230839 17032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:39:30.230988 17032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:39:30.251293 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "2a2871322b6749459cc6d571c97f1767"
format_stamp: "Formatted at 2026-06-20 11:39:30 on dist-test-slave-szdr"
server_key: "7264085c9352625fc71d5243abf5d3b1"
server_key_iv: "9a0a48dfd95946efa4fbfcde036093d6"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:30.251827 17032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "2a2871322b6749459cc6d571c97f1767"
format_stamp: "Formatted at 2026-06-20 11:39:30 on dist-test-slave-szdr"
server_key: "7264085c9352625fc71d5243abf5d3b1"
server_key_iv: "9a0a48dfd95946efa4fbfcde036093d6"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:30.253374 17032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:39:30.268262 17049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:30.268806 17032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 11:39:30.268894 17032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "2a2871322b6749459cc6d571c97f1767"
format_stamp: "Formatted at 2026-06-20 11:39:30 on dist-test-slave-szdr"
server_key: "7264085c9352625fc71d5243abf5d3b1"
server_key_iv: "9a0a48dfd95946efa4fbfcde036093d6"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:30.269011 17032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:39:30.286545 17032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:39:30.289180 17032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:39:30.289413 17032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:39:30.289696 17032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:39:30.290163 17032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:39:30.290221 17032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:30.290259 17032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:39:30.290282 17032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:30.297246 17032 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:40741
I20260620 11:39:30.297751 17032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:39:30.303010 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 17032
I20260620 11:39:30.303139 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:39:30.303367 5610 external_mini_cluster.cc:1428] Setting key 584e2276b9784875ed37786981dff99b
I20260620 11:39:30.306221 17163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:40741 every 8 connection(s)
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:39:30.326954 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.299074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:50757 (local address 127.5.122.190:45889)
0620 11:39:30.299125 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.299127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.306267 (+ 7140us) server_negotiation.cc:407] Connection header received
0620 11:39:30.307018 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.307020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.307037 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.307069 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:39:30.308014 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.308783 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.315292 (+ 6509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.315672 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.315738 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.323157 (+ 7419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:39:30.323159 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:39:30.323159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:39:30.323169 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:39:30.324946 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.325551 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.325553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.325554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.325634 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.326063 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.326064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.326064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.326231 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:39:30.326290 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.326720 (+ 430us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.326844 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:39:30.327863 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.298739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:45889 (local address 127.5.122.130:50757)
0620 11:39:30.306090 (+ 7351us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.306163 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.306871 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.307282 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.307288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.307573 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:39:30.307878 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.307886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.309175 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.309177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.310639 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.310642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.315865 (+ 5223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.315866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.316057 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.321136 (+ 5079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:39:30.321152 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:39:30.322981 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:39:30.325072 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:39:30.325086 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:39:30.325089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:39:30.325417 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:39:30.325730 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:39:30.325731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:39:30.325732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:39:30.325934 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:39:30.326332 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:39:30.326340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:39:30.326543 (+ 203us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.327352 (+ 809us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.327565 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7283,"thread_start_us":95,"threads_started":1}
I20260620 11:39:30.328719 17164 heartbeater.cc:347] Connected to a master server at 127.5.122.190:45889
I20260620 11:39:30.328846 17164 heartbeater.cc:464] Registering TS with master...
I20260620 11:39:30.329059 17164 heartbeater.cc:511] Master 127.5.122.190:45889 requested a full tablet report, sending...
I20260620 11:39:30.329481 16814 ts_manager.cc:194] Registered new tserver with Master: 2a2871322b6749459cc6d571c97f1767 (127.5.122.130:40741)
I20260620 11:39:30.330184 16814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:50757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:39:30.345582 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:33065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:45889
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:39:30.507481 17169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:39:30.507846 17169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:39:30.507930 17169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:39:30.509943 17169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:39:30.510082 17169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:39:30.510156 17169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:39:30.512637 17169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.122.148:33065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:45889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:39:30.513166 17169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:39:30.513499 17169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:39:30.520725 17174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:30.520752 17175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:39:30.521136 17177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:39:30.521658 17169 server_base.cc:1034] running on GCE node
I20260620 11:39:30.521900 17169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:39:30.522215 17169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:39:30.523393 17169 hybrid_clock.cc:648] HybridClock initialized: now 1781955570523367 us; error 47 us; skew 500 ppm
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:39:30.533615 17169 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:39:30.538254 17169 webserver.cc:460] Webserver started at http://127.5.122.131:43519/ using document root <none> and password file <none>
I20260620 11:39:30.538630 17169 fs_manager.cc:359] Metadata directory not provided
I20260620 11:39:30.538713 17169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:39:30.538870 17169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:39:30.554987 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "62c31637fee34b40b526dd21650b7a0e"
format_stamp: "Formatted at 2026-06-20 11:39:30 on dist-test-slave-szdr"
server_key: "9ce045d872dfe5eb54850931f11b41aa"
server_key_iv: "094ca51120a9ffa261e12729685401c2"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:30.555483 17169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "62c31637fee34b40b526dd21650b7a0e"
format_stamp: "Formatted at 2026-06-20 11:39:30 on dist-test-slave-szdr"
server_key: "9ce045d872dfe5eb54850931f11b41aa"
server_key_iv: "094ca51120a9ffa261e12729685401c2"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:30.560217 17169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 11:39:30.579653 17184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:30.579865 17169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:39:30.579975 17169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "62c31637fee34b40b526dd21650b7a0e"
format_stamp: "Formatted at 2026-06-20 11:39:30 on dist-test-slave-szdr"
server_key: "9ce045d872dfe5eb54850931f11b41aa"
server_key_iv: "094ca51120a9ffa261e12729685401c2"
server_key_version: "kuduclusterkey@0"
I20260620 11:39:30.580085 17169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:39:30.592994 17169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:39:30.595690 17169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:39:30.595862 17169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:39:30.596077 17169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:39:30.596462 17169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:39:30.596517 17169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:30.596561 17169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:39:30.596587 17169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:39:30.604048 17169 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:38611
I20260620 11:39:30.604722 17169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:39:30.605814 17297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:38611 every 8 connection(s)
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:39:30.614715 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 17169
I20260620 11:39:30.614838 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:39:30.615059 5610 external_mini_cluster.cc:1428] Setting key b6ca6ff258f5cfc17eaf231bdb316b80
I20260620 11:39:30.617770 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.606816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:34411 (local address 127.5.122.190:45889)
0620 11:39:30.606873 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.606876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.607223 (+ 347us) server_negotiation.cc:407] Connection header received
0620 11:39:30.607937 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.607938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.607950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.607982 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:39:30.608818 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.609577 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.611189 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.611468 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.611516 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.614295 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:39:30.614297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:39:30.614298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:39:30.614305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:39:30.616232 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.616781 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.616782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.616782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.616853 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.617088 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.617089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.617090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.617215 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:39:30.617301 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.617583 (+ 282us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.617677 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 11:39:30.618018 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.606480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:45889 (local address 127.5.122.131:34411)
0620 11:39:30.607088 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.607134 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.607795 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.608173 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.608178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.608431 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:39:30.608708 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.608714 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.609698 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.609699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.611080 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.611083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.611616 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.611617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.611767 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.612430 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:39:30.612441 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:39:30.614165 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:39:30.616328 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:39:30.616336 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:39:30.616338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:39:30.616697 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:39:30.616912 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:39:30.616913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:39:30.616913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:39:30.617014 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:39:30.617291 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:39:30.617297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:39:30.617441 (+ 144us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.617562 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.617730 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":130,"threads_started":1}
I20260620 11:39:30.619045 17298 heartbeater.cc:347] Connected to a master server at 127.5.122.190:45889
I20260620 11:39:30.619172 17298 heartbeater.cc:464] Registering TS with master...
I20260620 11:39:30.619374 17298 heartbeater.cc:511] Master 127.5.122.190:45889 requested a full tablet report, sending...
I20260620 11:39:30.619751 16814 ts_manager.cc:194] Registered new tserver with Master: 62c31637fee34b40b526dd21650b7a0e (127.5.122.131:38611)
I20260620 11:39:30.620239 16814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:34411
I20260620 11:39:30.627801 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:39:30.637876 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.629627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.5.122.190:45889)
0620 11:39:30.629694 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.629697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.630049 (+ 352us) server_negotiation.cc:407] Connection header received
0620 11:39:30.630166 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.630167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.630191 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.630223 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:39:30.631012 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.631530 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.632431 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.632632 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.632720 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.634433 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:39:30.634434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:39:30.634435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:39:30.634444 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:39:30.636406 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.636864 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.636865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.636865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.636944 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:39:30.637175 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:39:30.637176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:39:30.637176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:39:30.637338 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:39:30.637427 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.637685 (+ 258us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.637775 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 11:39:30.640427 16814 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:39:30.642182 16814 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:39:30.653034 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.645746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:38611 (local address 127.0.0.1:42958)
0620 11:39:30.647143 (+ 1397us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.647194 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.647263 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.648369 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.648369 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.648372 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:39:30.648410 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.648411 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.649805 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.649806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.651591 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.651593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.652737 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.652738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.652814 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.652828 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.652893 (+ 65us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.652945 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1358,"mutex_wait_us":26,"thread_start_us":143,"threads_started":1}
I20260620 11:39:30.653391 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.646453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.5.122.131:38611)
0620 11:39:30.647535 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.647542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.647553 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:39:30.647594 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.647596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.647613 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.647670 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:39:30.648499 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.649704 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.651699 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.652552 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.652626 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.652703 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.653261 (+ 558us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.653303 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1041,"thread_start_us":81,"threads_started":1}
I20260620 11:39:30.654182 17232 tablet_service.cc:1467] Processing CreateTablet for tablet 5631b61123ea46a2a8a98714a3846142 (DEFAULT_TABLE table=test-table [id=0a1df5559c344dfeb50e8930bf52827b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:39:30.654600 17232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5631b61123ea46a2a8a98714a3846142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:39:30.654903 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.646929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:39903 (local address 127.0.0.1:40752)
0620 11:39:30.649206 (+ 2277us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.649217 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.649260 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.650394 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.650395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.650397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:39:30.650432 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.650434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.652013 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.652014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.653082 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.653084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.654558 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.654559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.654624 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.654630 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.654772 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.654818 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2258,"thread_start_us":67,"threads_started":1}
I20260620 11:39:30.656232 17315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.647043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.5.122.129:39903)
0620 11:39:30.650173 (+ 3130us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.650177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.650188 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:39:30.650223 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.650225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.650241 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.650319 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:39:30.650507 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.651922 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.653597 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.654451 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.655986 (+ 1535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.656064 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.656096 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.656144 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3092,"thread_start_us":54,"threads_started":1}
I20260620 11:39:30.657045 16960 tablet_service.cc:1467] Processing CreateTablet for tablet 5631b61123ea46a2a8a98714a3846142 (DEFAULT_TABLE table=test-table [id=0a1df5559c344dfeb50e8930bf52827b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:39:30.657404 16960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5631b61123ea46a2a8a98714a3846142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:39:30.661320 17317 tablet_bootstrap.cc:492] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6: Bootstrap starting.
I20260620 11:39:30.661979 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.646083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:40741 (local address 127.0.0.1:58022)
0620 11:39:30.647324 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.647335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.647386 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.648065 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.648069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.648071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:39:30.648149 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.648152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.649949 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.649950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.660084 (+ 10134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.660090 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.661271 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.661272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.661360 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.661366 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.661432 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.661874 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1227,"mutex_wait_us":43,"thread_start_us":63,"threads_started":1}
I20260620 11:39:30.662207 17317 tablet_bootstrap.cc:654] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6: Neither blocks nor log segments found. Creating new log.
I20260620 11:39:30.662577 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.646619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.5.122.130:40741)
0620 11:39:30.647828 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.647832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.647843 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:39:30.647880 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.647882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.647897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.647974 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:39:30.648240 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.649863 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.660246 (+ 10383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.661076 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.661150 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.661229 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.662407 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.662471 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1167,"thread_start_us":75,"threads_started":1}
I20260620 11:39:30.662599 17317 log.cc:824] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6: Log is configured to *not* fsync() on all Append() calls
I20260620 11:39:30.663291 17098 tablet_service.cc:1467] Processing CreateTablet for tablet 5631b61123ea46a2a8a98714a3846142 (DEFAULT_TABLE table=test-table [id=0a1df5559c344dfeb50e8930bf52827b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:39:30.663673 17098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5631b61123ea46a2a8a98714a3846142. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:39:30.663676 17317 tablet_bootstrap.cc:492] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6: No bootstrap required, opened a new log
I20260620 11:39:30.663741 17317 ts_tablet_manager.cc:1397] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:39:30.666571 17317 raft_consensus.cc:348] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.666774 17317 raft_consensus.cc:374] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:39:30.666812 17317 raft_consensus.cc:729] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5566e12633364bb7b1ac45e4543790c6, State: Initialized, Role: FOLLOWER
I20260620 11:39:30.667256 17317 consensus_queue.cc:260] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.667701 17317 ts_tablet_manager.cc:1428] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 11:39:30.667924 17026 heartbeater.cc:503] Master 127.5.122.190:45889 was elected leader, sending a full tablet report...
I20260620 11:39:30.668236 17316 tablet_bootstrap.cc:492] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Bootstrap starting.
I20260620 11:39:30.669394 17316 tablet_bootstrap.cc:654] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Neither blocks nor log segments found. Creating new log.
I20260620 11:39:30.669728 17316 log.cc:824] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Log is configured to *not* fsync() on all Append() calls
I20260620 11:39:30.670123 17320 tablet_bootstrap.cc:492] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Bootstrap starting.
I20260620 11:39:30.671141 17320 tablet_bootstrap.cc:654] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Neither blocks nor log segments found. Creating new log.
I20260620 11:39:30.672149 17316 tablet_bootstrap.cc:492] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: No bootstrap required, opened a new log
I20260620 11:39:30.672223 17316 ts_tablet_manager.cc:1397] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260620 11:39:30.674479 17320 log.cc:824] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Log is configured to *not* fsync() on all Append() calls
I20260620 11:39:30.675407 17316 raft_consensus.cc:348] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.675618 17316 raft_consensus.cc:374] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:39:30.675664 17316 raft_consensus.cc:729] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c31637fee34b40b526dd21650b7a0e, State: Initialized, Role: FOLLOWER
I20260620 11:39:30.675817 17316 consensus_queue.cc:260] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.676144 17316 ts_tablet_manager.cc:1428] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 11:39:30.676275 17298 heartbeater.cc:503] Master 127.5.122.190:45889 was elected leader, sending a full tablet report...
I20260620 11:39:30.677762 17320 tablet_bootstrap.cc:492] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: No bootstrap required, opened a new log
I20260620 11:39:30.677918 17320 ts_tablet_manager.cc:1397] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260620 11:39:30.679466 17320 raft_consensus.cc:348] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.679651 17320 raft_consensus.cc:374] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:39:30.679697 17320 raft_consensus.cc:729] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a2871322b6749459cc6d571c97f1767, State: Initialized, Role: FOLLOWER
I20260620 11:39:30.679840 17320 consensus_queue.cc:260] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.680171 17320 ts_tablet_manager.cc:1428] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:39:30.680313 17164 heartbeater.cc:503] Master 127.5.122.190:45889 was elected leader, sending a full tablet report...
I20260620 11:39:30.684538 17319 raft_consensus.cc:482] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:39:30.684684 17319 raft_consensus.cc:504] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.685125 17319 leader_election.cc:288] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62c31637fee34b40b526dd21650b7a0e (127.5.122.131:38611), 2a2871322b6749459cc6d571c97f1767 (127.5.122.130:40741)
I20260620 11:39:30.694188 17324 raft_consensus.cc:482] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:39:30.694337 17324 raft_consensus.cc:504] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.699992 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.695397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:39903 (local address 127.5.122.130:53269)
0620 11:39:30.695456 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.695467 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.695521 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.695706 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.695707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.695708 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:39:30.695782 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.695784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.697186 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.697187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.698756 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.698761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.699694 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.699695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.699789 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.699794 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.699844 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.699891 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 11:39:30.700347 17315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.695224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:53269 (local address 127.5.122.129:39903)
0620 11:39:30.695347 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.695349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.695576 (+ 227us) server_negotiation.cc:407] Connection header received
0620 11:39:30.695605 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.695607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.695617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.695660 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:39:30.695857 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.696670 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.698887 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.699543 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.699634 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.699645 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.700209 (+ 564us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.700257 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260620 11:39:30.700806 16980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5631b61123ea46a2a8a98714a3846142" candidate_uuid: "2a2871322b6749459cc6d571c97f1767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5566e12633364bb7b1ac45e4543790c6" is_pre_election: true
I20260620 11:39:30.700941 16980 raft_consensus.cc:2454] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2871322b6749459cc6d571c97f1767 in term 0.
I20260620 11:39:30.701329 17324 leader_election.cc:288] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5566e12633364bb7b1ac45e4543790c6 (127.5.122.129:39903), 62c31637fee34b40b526dd21650b7a0e (127.5.122.131:38611)
I20260620 11:39:30.701434 17053 leader_election.cc:302] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2871322b6749459cc6d571c97f1767, 5566e12633364bb7b1ac45e4543790c6; no voters:
I20260620 11:39:30.701650 17324 raft_consensus.cc:2781] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:39:30.701772 17324 raft_consensus.cc:482] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:39:30.701856 17324 raft_consensus.cc:3037] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:39:30.702886 17324 raft_consensus.cc:504] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.703346 16980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5631b61123ea46a2a8a98714a3846142" candidate_uuid: "2a2871322b6749459cc6d571c97f1767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5566e12633364bb7b1ac45e4543790c6"
I20260620 11:39:30.703459 16980 raft_consensus.cc:3037] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:39:30.704355 16980 raft_consensus.cc:2454] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2871322b6749459cc6d571c97f1767 in term 1.
I20260620 11:39:30.704622 17053 leader_election.cc:302] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a2871322b6749459cc6d571c97f1767, 5566e12633364bb7b1ac45e4543790c6; no voters:
I20260620 11:39:30.704914 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.700975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:38611 (local address 127.5.122.130:50611)
0620 11:39:30.701226 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.701237 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.701274 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.701698 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.701699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.701701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:39:30.701746 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.701747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.702863 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.702864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.703871 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.703875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.704675 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.704676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.704743 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.704748 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.704782 (+ 34us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.704829 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260620 11:39:30.704939 17324 leader_election.cc:288] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [CANDIDATE]: Term 1 election: Requested vote from peers 5566e12633364bb7b1ac45e4543790c6 (127.5.122.129:39903), 62c31637fee34b40b526dd21650b7a0e (127.5.122.131:38611)
I20260620 11:39:30.704993 17324 raft_consensus.cc:2781] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:39:30.705037 17324 raft_consensus.cc:686] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 1 LEADER]: Becoming Leader. State: Replica: 2a2871322b6749459cc6d571c97f1767, State: Running, Role: LEADER
I20260620 11:39:30.705240 17324 consensus_queue.cc:237] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } }
I20260620 11:39:30.705394 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.701080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:50611 (local address 127.5.122.131:38611)
0620 11:39:30.701548 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.701551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.701559 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:39:30.701587 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.701588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.701599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.701810 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:39:30.701851 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.702739 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.703998 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.704581 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.705197 (+ 616us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.705208 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.705229 (+ 21us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.705274 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":108,"threads_started":1}
I20260620 11:39:30.705756 17250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5631b61123ea46a2a8a98714a3846142" candidate_uuid: "2a2871322b6749459cc6d571c97f1767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c31637fee34b40b526dd21650b7a0e" is_pre_election: true
I20260620 11:39:30.705929 17250 raft_consensus.cc:2454] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a2871322b6749459cc6d571c97f1767 in term 0.
I20260620 11:39:30.706085 17249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5631b61123ea46a2a8a98714a3846142" candidate_uuid: "2a2871322b6749459cc6d571c97f1767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c31637fee34b40b526dd21650b7a0e"
I20260620 11:39:30.706156 17249 raft_consensus.cc:3037] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:39:30.707093 17249 raft_consensus.cc:2454] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a2871322b6749459cc6d571c97f1767 in term 1.
I20260620 11:39:30.707617 16814 catalog_manager.cc:5495] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a2871322b6749459cc6d571c97f1767 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "2a2871322b6749459cc6d571c97f1767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a2871322b6749459cc6d571c97f1767" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 40741 } health_report { overall_health: HEALTHY } } }
Jun 20 11:39:30 dist-test-slave-szdr krb5kdc[16338](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955570, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:39:30.713629 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.686342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:50891 (local address 127.5.122.130:40741)
0620 11:39:30.686472 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.686474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.700554 (+ 14080us) server_negotiation.cc:407] Connection header received
0620 11:39:30.705981 (+ 5427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.705983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.705997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.706027 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:39:30.708696 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.709369 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.711751 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.712336 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.712378 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.712388 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.713498 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.713548 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260620 11:39:30.714170 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.685674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:40741 (local address 127.5.122.129:50891)
0620 11:39:30.700489 (+ 14815us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.700504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.700586 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.706237 (+ 5651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.706238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.706242 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:39:30.706330 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.706333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.709642 (+ 3309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.709643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.710732 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.710735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.712465 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.712465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.712541 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.712545 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.714008 (+ 1463us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.714072 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14798,"thread_start_us":58,"threads_started":1}
I20260620 11:39:30.714664 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.685824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:58651 (local address 127.5.122.131:38611)
0620 11:39:30.685875 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.685878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.700429 (+ 14551us) server_negotiation.cc:407] Connection header received
0620 11:39:30.706123 (+ 5694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.706123 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.706135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.706164 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:39:30.706585 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.707221 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.712643 (+ 5422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.713191 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.713238 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.713245 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.714535 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.714596 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:39:30.715633 17331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.711577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.5.122.130:40741)
0620 11:39:30.711734 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0620 11:39:30.711736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:39:30.711878 (+ 142us) server_negotiation.cc:407] Connection header received
0620 11:39:30.711925 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:39:30.711925 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:39:30.711946 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:39:30.712179 (+ 233us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:39:30.712218 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.712995 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.714177 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.714499 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.714731 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.714802 (+ 71us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:39:30.714964 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:39:30.715506 (+ 542us) server_negotiation.cc:1035] Waiting for connection context
0620 11:39:30.715528 (+ 22us) server_negotiation.cc:299] Negotiation successful
0620 11:39:30.715549 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260620 11:39:30.717025 17110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5631b61123ea46a2a8a98714a3846142" candidate_uuid: "5566e12633364bb7b1ac45e4543790c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a2871322b6749459cc6d571c97f1767" is_pre_election: true
I20260620 11:39:30.717746 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.685433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:38611 (local address 127.5.122.129:58651)
0620 11:39:30.700338 (+ 14905us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.700351 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:39:30.700643 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:39:30.706442 (+ 5799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:39:30.706443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:39:30.706446 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:39:30.706507 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.706508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.707327 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.707328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.709161 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:39:30.709166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:39:30.713306 (+ 4140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:39:30.713307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:39:30.713395 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:39:30.713399 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:39:30.715815 (+ 2416us) client_negotiation.cc:240] Negotiation successful
0620 11:39:30.717617 (+ 1802us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14884,"thread_start_us":129,"threads_started":1}
I20260620 11:39:30.718221 17249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5631b61123ea46a2a8a98714a3846142" candidate_uuid: "5566e12633364bb7b1ac45e4543790c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62c31637fee34b40b526dd21650b7a0e" is_pre_election: true
I20260620 11:39:30.718381 17249 raft_consensus.cc:2379] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5566e12633364bb7b1ac45e4543790c6 in current term 1: Already voted for candidate 2a2871322b6749459cc6d571c97f1767 in this term.
I20260620 11:39:30.718673 16913 leader_election.cc:302] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5566e12633364bb7b1ac45e4543790c6; no voters: 2a2871322b6749459cc6d571c97f1767, 62c31637fee34b40b526dd21650b7a0e
I20260620 11:39:30.718945 17319 raft_consensus.cc:2726] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260620 11:39:30.723093 16980 raft_consensus.cc:1261] T 5631b61123ea46a2a8a98714a3846142 P 5566e12633364bb7b1ac45e4543790c6 [term 1 FOLLOWER]: Refusing update from remote peer 2a2871322b6749459cc6d571c97f1767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:39:30.723441 17249 raft_consensus.cc:1261] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 1 FOLLOWER]: Refusing update from remote peer 2a2871322b6749459cc6d571c97f1767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:39:30.723739 17324 consensus_queue.cc:1035] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5566e12633364bb7b1ac45e4543790c6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 39903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:39:30.723937 17324 consensus_queue.cc:1035] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62c31637fee34b40b526dd21650b7a0e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:39:30.731693 17332 mvcc.cc:205] Tried to move back new op lower bound from 7298890017677852672 to 7298890017609228288. Current Snapshot: MvccSnapshot[applied={T|T < 7298890017677852672}]
I20260620 11:39:30.732591 17333 mvcc.cc:205] Tried to move back new op lower bound from 7298890017677852672 to 7298890017609228288. Current Snapshot: MvccSnapshot[applied={T|T < 7298890017677852672}]
I20260620 11:39:30.740310 17335 mvcc.cc:205] Tried to move back new op lower bound from 7298890017677852672 to 7298890017609228288. Current Snapshot: MvccSnapshot[applied={T|T < 7298890017677852672}]
I20260620 11:39:30.742568 16814 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:39:30.742687 16814 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:39:30.743476 16814 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caf1a1bc82f84fb6bccaf3896341112c: Sending DeleteTablet for 3 replicas of tablet 5631b61123ea46a2a8a98714a3846142
I20260620 11:39:30.743914 17232 tablet_service.cc:1514] Processing DeleteTablet for tablet 5631b61123ea46a2a8a98714a3846142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:39:30 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:42958
I20260620 11:39:30.744340 16960 tablet_service.cc:1514] Processing DeleteTablet for tablet 5631b61123ea46a2a8a98714a3846142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:39:30 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:40752
I20260620 11:39:30.744724 17098 tablet_service.cc:1514] Processing DeleteTablet for tablet 5631b61123ea46a2a8a98714a3846142 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:39:30 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58022
I20260620 11:39:30.745247 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 16895
I20260620 11:39:30.754423 17341 tablet_replica.cc:330] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: stopping tablet replica
I20260620 11:39:30.754606 17341 raft_consensus.cc:2229] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:39:30.754748 17341 raft_consensus.cc:2258] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:39:30.756533 17341 ts_tablet_manager.cc:1905] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:39:30.758844 17343 tablet_replica.cc:330] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: stopping tablet replica
I20260620 11:39:30.759063 17343 raft_consensus.cc:2229] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:39:30.762530 17343 raft_consensus.cc:2258] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:39:30.763266 17343 ts_tablet_manager.cc:1905] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:39:30.758503 17341 ts_tablet_manager.cc:1918] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:39:30.764752 17341 log.cc:1196] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/5631b61123ea46a2a8a98714a3846142
I20260620 11:39:30.765218 17341 ts_tablet_manager.cc:1939] T 5631b61123ea46a2a8a98714a3846142 P 62c31637fee34b40b526dd21650b7a0e: Deleting consensus metadata
I20260620 11:39:30.766865 16798 catalog_manager.cc:4841] TS 62c31637fee34b40b526dd21650b7a0e (127.5.122.131:38611): tablet 5631b61123ea46a2a8a98714a3846142 (table test-table [id=0a1df5559c344dfeb50e8930bf52827b]) successfully deleted
I20260620 11:39:30.767213 17343 ts_tablet_manager.cc:1918] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:39:30.767560 17343 log.cc:1196] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/5631b61123ea46a2a8a98714a3846142
W20260620 11:39:30.767907 16801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:39903 (error 108)
I20260620 11:39:30.768288 17343 ts_tablet_manager.cc:1939] T 5631b61123ea46a2a8a98714a3846142 P 2a2871322b6749459cc6d571c97f1767: Deleting consensus metadata
I20260620 11:39:30.768929 16798 catalog_manager.cc:4841] TS 2a2871322b6749459cc6d571c97f1767 (127.5.122.130:40741): tablet 5631b61123ea46a2a8a98714a3846142 (table test-table [id=0a1df5559c344dfeb50e8930bf52827b]) successfully deleted
I20260620 11:39:30.769196 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 17032
I20260620 11:39:30.771690 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:39:30.771538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:39903 (local address 127.0.0.1:40766)
0620 11:39:30.771597 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:39:30.771653 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:39903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260620 11:39:30.771844 16801 catalog_manager.cc:4584] TS 5566e12633364bb7b1ac45e4543790c6 (127.5.122.129:39903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5631b61123ea46a2a8a98714a3846142: Network error: Client connection negotiation failed: client connection to 127.5.122.129:39903: connect: Connection refused (error 111)
I20260620 11:39:30.776300 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 17169
I20260620 11:39:30.782908 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 16782
I20260620 11:39:30.792218 5610 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 11:39:31.137809 5610 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 11:39:31.137925 5610 mini_ranger.cc:66] Stopping Ranger...
I20260620 11:39:31.483253 5610 mini_ranger.cc:68] Stopped Ranger
2026-06-20 11:39:31.483 UTC [16368] LOG: received smart shutdown request
2026-06-20 11:39:31.487 UTC [16368] LOG: background worker "logical replication launcher" (PID 16377) exited with exit code 1
2026-06-20 11:39:31.487 UTC [16372] LOG: shutting down
2026-06-20 11:39:31.680 UTC [16368] LOG: database system is shut down
2026-06-20T11:39:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49063 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:39:31 dist-test-slave-szdr krb5kdc[17353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:39:31 dist-test-slave-szdr krb5kdc[17353](info): set up 2 sockets
Jun 20 11:39:31 dist-test-slave-szdr krb5kdc[17353](info): commencing operation
krb5kdc: starting...
W20260620 11:39:32.906591 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:39:32 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:39:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:39:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/rangeradmin_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/rangeradmin_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/rangerlookup_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/rangerlookup_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
I20260620 11:39:33.089336 5610 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJIe33o/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/postgres -l logfile start
2026-06-20 11:39:35.872 UTC [17383] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 11:39:35.872 UTC [17383] LOG: listening on IPv4 address "127.5.122.148", port 42905
2026-06-20 11:39:35.876 UTC [17383] LOG: listening on Unix socket "/tmp/.s.PGSQL.42905"
2026-06-20 11:39:35.900 UTC [17386] LOG: database system was shut down at 2026-06-20 11:39:34 UTC
2026-06-20 11:39:35.905 UTC [17383] LOG: database system is ready to accept connections
I20260620 11:39:36.866339 5610 mini_postgres.cc:96] Postgres bound to 42905
2026-06-20 11:39:36.872 UTC [17394] FATAL: database "slave" does not exist
127.5.122.148:42905 - accepting connections
I20260620 11:39:36.873606 5610 mini_ranger.cc:161] Starting Ranger...
I20260620 11:39:36.890184 5610 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 11:39:37.470439 5610 mini_ranger.cc:87] Created ranger Postgres database
I20260620 11:39:37.470541 5610 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-admin
2026-06-20 11:39:37,761 [I] DB FLAVOR :POSTGRES
2026-06-20 11:39:37,762 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 11:39:37,762 [I] Checking connection..
2026-06-20 11:39:37,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 11:39:38,031 [I] Checking connection passed.
2026-06-20 11:39:38,031 [I] --------- Verifying version history table ---------
2026-06-20 11:39:38,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 11:39:38,307 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 11:39:38,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 11:39:38,573 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 11:39:38,573 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 11:39:38,573 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 11:39:38.837 UTC [17458] WARNING: there is no transaction in progress
2026-06-20 11:39:38,845 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 11:39:38,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 11:39:39,111 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 11:39:39,111 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 11:39:39,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 11:39:39,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 11:39:39,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-szdr.c.gcp-upstream.internal','N') ;"
2026-06-20 11:39:39,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 11:39:40,136 [I] Table x_portal_user does not exist in database ranger
2026-06-20 11:39:40,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 11:39:40,387 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 11:39:40,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 11:39:40,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 11:39:40,913 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 11:39:40,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 11:39:42.065 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.085 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.105 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.124 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.143 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.164 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.237 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.247 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.265 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.279 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.292 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.311 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.323 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.332 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.340 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:42.348 UTC [17566] WARNING: there is no transaction in progress
2026-06-20 11:39:43,175 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 11:39:43,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 11:39:43,414 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 11:39:43,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 11:39:43,677 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 11:39:43,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 11:39:43,953 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 11:39:43,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 11:39:44,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 11:39:44,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-szdr.c.gcp-upstream.internal';"
2026-06-20 11:39:44,716 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 11:39:44,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 11:39:44,998 [I] DB_PATCHES have already been applied
I20260620 11:39:45.004493 5610 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJIe33o/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 11:39:45.004567 5610 mini_ranger.cc:193] Using host: 127.5.122.148
I20260620 11:39:45.007838 5610 mini_ranger.cc:227] Ranger admin URL: http://127.5.122.148:44007
Jun 20, 2026 11:39:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:39:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:39:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:39:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:39:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:39:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:39:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 11:39:46.144469 5610 mini_ranger.cc:160] Time spent starting Ranger: real 9.271s user 0.006s sys 0.004s
Jun 20, 2026 11:40:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:40:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 20, 2026 11:40:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:40:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:40:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 11:40:08.776347 5610 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.122.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.122.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/rangerkms_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.122.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/rangerkms_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.122.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.122.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.122.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/HTTP_127.5.122.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 11:40:08 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955608, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 11:40:08.866990 5610 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 11:40:08.877594 5610 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 11:40:09.885283 5610 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 11:40:09.885386 5610 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms
I20260620 11:40:09.885411 5610 mini_ranger_kms.cc:227] Using postgres at 127.5.122.148:42905
2026-06-20 11:40:10,059 [I] DB FLAVOR :POSTGRES
2026-06-20 11:40:10,059 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 11:40:10,059 [I] Checking connection
2026-06-20 11:40:10,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 11:40:10,320 [I] connection success
2026-06-20 11:40:10,320 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 11:40:10,320 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 11:40:10,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 11:40:10,571 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 11:40:10,571 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 11:40:10,572 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 11:40:10,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.122.148:42905/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 11:40:10,856 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 11:40:11.405122 5610 mini_ranger_kms.cc:310] Created kms service
I20260620 11:40:11.621282 5610 mini_ranger_kms.cc:326] Created kudu user
I20260620 11:40:11.675858 5610 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 11:40:11 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955608, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:12.159847 5610 mini_ranger_kms.cc:384] Added ranger policy
I20260620 11:40:12.160130 5610 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJIe33o/build/release/bin/postgresql.jar:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJIe33o/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 11:40:12.160161 5610 mini_ranger_kms.cc:240] Using host: 127.5.122.148
I20260620 11:40:12.163977 5610 mini_ranger_kms.cc:276] Ranger KMS PID: 17747
I20260620 11:40:12.164088 5610 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.122.148:56157
Jun 20, 2026 11:40:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:40:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:40:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:40:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:40:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:40:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 11:40:14.408308 5610 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.541s user 0.002s sys 0.155s
I20260620 11:40:14.408433 5610 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 11:40:14.408481 5610 mini_ranger_kms.cc:401] 127.5.122.148:56157/kms/v1/keys
Jun 20 11:40:14 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781955614, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 11:40:15 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955614, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.148@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
Jun 20 11:40:15 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955614, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.122.148@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
Jun 20 11:40:17 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:40:17.869009 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35077
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:35077
--ranger_config_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:17.955730 17797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:17.955993 17797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:17.956018 17797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:17.957378 17797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:17.957442 17797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:17.957458 17797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:17.957470 17797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:17.957482 17797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:17.959434 17797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.122.148:56157/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:35077
--ranger_config_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:17.959800 17797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:17.960096 17797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:17.966777 17802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:17.966780 17805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:17.966780 17803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:17.967181 17797 server_base.cc:1034] running on GCE node
I20260620 11:40:17.967476 17797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:17.967823 17797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:17.969015 17797 hybrid_clock.cc:648] HybridClock initialized: now 1781955617968999 us; error 35 us; skew 500 ppm
Jun 20 11:40:17 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955617, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:17.971895 17797 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:17.972498 17797 webserver.cc:460] Webserver started at http://127.5.122.190:36389/ using document root <none> and password file <none>
I20260620 11:40:17.972762 17797 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:17.972827 17797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:17.972988 17797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:40:17 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955617, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:18.033818 17797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "d09670ae27a24184961918bf1e14a49b"
format_stamp: "Formatted at 2026-06-20 11:40:18 on dist-test-slave-szdr"
server_key: "d65e984cad4d858407f294bd5dc32009"
server_key_iv: "7ada3450743d9541dce207817c28248d"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:18.034420 17797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "d09670ae27a24184961918bf1e14a49b"
format_stamp: "Formatted at 2026-06-20 11:40:18 on dist-test-slave-szdr"
server_key: "d65e984cad4d858407f294bd5dc32009"
server_key_iv: "7ada3450743d9541dce207817c28248d"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:18.036187 17797 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 11:40:18.053731 17812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:18.054045 17797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:18.054167 17797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "d09670ae27a24184961918bf1e14a49b"
format_stamp: "Formatted at 2026-06-20 11:40:18 on dist-test-slave-szdr"
server_key: "d65e984cad4d858407f294bd5dc32009"
server_key_iv: "7ada3450743d9541dce207817c28248d"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:18.054293 17797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:18.064839 17797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:18.067373 17797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:18.067544 17797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:18.073084 17797 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:35077
I20260620 11:40:18.073118 17864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:35077 every 8 connection(s)
I20260620 11:40:18.073565 17797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:18.074465 17865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:18.075938 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 17797
I20260620 11:40:18.076078 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:40:18.076328 5610 external_mini_cluster.cc:1428] Setting key fc74b2668767afae2dd8be9777e90a23
I20260620 11:40:18.078136 17865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Bootstrap starting.
I20260620 11:40:18.078966 17865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:18.079545 17865 log.cc:824] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:18.080431 17865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: No bootstrap required, opened a new log
I20260620 11:40:18.082856 17865 raft_consensus.cc:348] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:18.083072 17865 raft_consensus.cc:374] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:18.083141 17865 raft_consensus.cc:729] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d09670ae27a24184961918bf1e14a49b, State: Initialized, Role: FOLLOWER
I20260620 11:40:18.083328 17865 consensus_queue.cc:260] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:18.083431 17865 raft_consensus.cc:388] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:40:18.083488 17865 raft_consensus.cc:482] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:40:18.083559 17865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:18.084342 17865 raft_consensus.cc:504] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:18.084493 17865 leader_election.cc:302] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d09670ae27a24184961918bf1e14a49b; no voters:
I20260620 11:40:18.084721 17865 leader_election.cc:288] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:40:18.084753 17870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:18.084995 17870 raft_consensus.cc:686] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 LEADER]: Becoming Leader. State: Replica: d09670ae27a24184961918bf1e14a49b, State: Running, Role: LEADER
I20260620 11:40:18.085127 17865 sys_catalog.cc:564] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:18.085135 17870 consensus_queue.cc:237] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:18.085757 17870 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d09670ae27a24184961918bf1e14a49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } } }
I20260620 11:40:18.085856 17870 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:18.086066 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d09670ae27a24184961918bf1e14a49b. Latest consensus state: current_term: 1 leader_uuid: "d09670ae27a24184961918bf1e14a49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } } }
I20260620 11:40:18.086190 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: This master's current role is: LEADER
Jun 20 11:40:18 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:18.086887 17879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:40:18.087487 17879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:18.087828 17865 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 11:40:18.090457 17879 catalog_manager.cc:1347] Generated new cluster ID: c69bcf00ad42459b92bfe6a3286cc23b
I20260620 11:40:18.090583 17879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:40:18.091212 17868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:18.076989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.5.122.190:35077)
0620 11:40:18.078155 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:18.078164 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:18.078195 (+ 31us) server_negotiation.cc:407] Connection header received
0620 11:40:18.078731 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:18.078740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:18.078797 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:18.079145 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:18.079992 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:18.081518 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:18.082722 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:18.083400 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:18.083521 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:18.086897 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:18.086902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:18.086902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:18.086916 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:18.089032 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:18.089486 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:18.089489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:18.089490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:18.089593 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:18.089807 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:18.089807 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:18.089808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:18.090164 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:18.090326 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:18.090819 (+ 493us) server_negotiation.cc:299] Negotiation successful
0620 11:40:18.090951 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":82,"threads_started":1}
I20260620 11:40:18.101574 17879 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:40:18.102504 17879 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:40:18.116185 17879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Generated new TSK 0
I20260620 11:40:18.116468 17879 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:40:18.901916 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35077
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:19.000046 17910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:19.000449 17910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:19.000519 17910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:19.001777 17910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:19.001852 17910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:19.001897 17910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:40:19.003571 17910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.122.148:56157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:19.003829 17910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:19.004099 17910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:19.010957 17916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:19.011010 17910 server_base.cc:1034] running on GCE node
W20260620 11:40:19.011148 17918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:19.011209 17915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:19.011439 17910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:19.011762 17910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:19.012940 17910 hybrid_clock.cc:648] HybridClock initialized: now 1781955619012915 us; error 41 us; skew 500 ppm
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:19.015904 17910 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:40:19.016443 17910 webserver.cc:460] Webserver started at http://127.5.122.129:42341/ using document root <none> and password file <none>
I20260620 11:40:19.016721 17910 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:19.016795 17910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:19.016963 17910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:19.032896 17910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "d215537622bf4c9ea7a2cb84d7bcc717"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "67051089d064575b41c2ef27bda9bd99"
server_key_iv: "74d9e15ab57c455196b771d837808448"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.033453 17910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "d215537622bf4c9ea7a2cb84d7bcc717"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "67051089d064575b41c2ef27bda9bd99"
server_key_iv: "74d9e15ab57c455196b771d837808448"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.038519 17910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260620 11:40:19.053200 17925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.053485 17910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:19.053603 17910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "d215537622bf4c9ea7a2cb84d7bcc717"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "67051089d064575b41c2ef27bda9bd99"
server_key_iv: "74d9e15ab57c455196b771d837808448"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.053709 17910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:19.083904 17910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:19.086702 17910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:19.086949 17910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:19.087260 17910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:19.087669 17910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:19.087731 17910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.087800 17910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:19.087833 17910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.095366 17910 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:33677
I20260620 11:40:19.095818 17910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:40:19.096359 18040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:33677 every 8 connection(s)
I20260620 11:40:19.099886 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 17910
I20260620 11:40:19.099997 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:40:19.100204 5610 external_mini_cluster.cc:1428] Setting key 4d2f3aa3fa4e7d716be8c50d978397b3
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:19.109694 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:19.096994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:52441 (local address 127.5.122.190:35077)
0620 11:40:19.098223 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:19.098225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:19.098234 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:40:19.098269 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:19.098269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:19.098281 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:19.098375 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:19.099018 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.099561 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.101018 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.101341 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.101402 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:19.105704 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:19.105706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:19.105707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:19.105716 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:19.107918 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:19.108511 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:19.108513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:19.108513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:19.108587 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:19.108880 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:19.108881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:19.108881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:19.109058 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:19.109133 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:19.109484 (+ 351us) server_negotiation.cc:299] Negotiation successful
0620 11:40:19.109579 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1184,"thread_start_us":80,"threads_started":1}
I20260620 11:40:19.110105 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:19.096626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35077 (local address 127.5.122.129:52441)
0620 11:40:19.097166 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:19.097234 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:19.097919 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:19.098470 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:19.098474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:19.098695 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:19.098906 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:19.098911 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.099666 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.099667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:19.100923 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:19.100926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.101489 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.101490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:19.101654 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:19.103566 (+ 1912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:19.103579 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:19.105534 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:19.108034 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:19.108044 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:19.108047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:19.108412 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:19.108673 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:19.108673 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:19.108674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:19.108806 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:19.109161 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:19.109168 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:19.109331 (+ 163us) client_negotiation.cc:769] Sending connection context
0620 11:40:19.109450 (+ 119us) client_negotiation.cc:240] Negotiation successful
0620 11:40:19.109621 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":81,"threads_started":1}
I20260620 11:40:19.110667 18041 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35077
I20260620 11:40:19.110795 18041 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:19.110988 18041 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:19.111572 17829 ts_manager.cc:194] Registered new tserver with Master: d215537622bf4c9ea7a2cb84d7bcc717 (127.5.122.129:33677)
I20260620 11:40:19.112208 17829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:52441
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:40:19.137768 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35077
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:19.261610 18047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:19.261952 18047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:19.262032 18047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:19.263960 18047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:19.264045 18047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:19.264091 18047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:40:19.266603 18047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.122.148:56157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:19.266934 18047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:19.267246 18047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 11:40:19.274485 18055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:19.274509 18054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:19.274935 18057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:19.276283 18047 server_base.cc:1034] running on GCE node
I20260620 11:40:19.276594 18047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:19.276976 18047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:19.278180 18047 hybrid_clock.cc:648] HybridClock initialized: now 1781955619278145 us; error 50 us; skew 500 ppm
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:19.282366 18047 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:40:19.283016 18047 webserver.cc:460] Webserver started at http://127.5.122.130:43731/ using document root <none> and password file <none>
I20260620 11:40:19.283306 18047 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:19.283367 18047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:19.283584 18047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:19.312289 18047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "cb97544e596d4deba3f18046c96c0096"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "4c14c206fc082317973bed96606aef95"
server_key_iv: "1f398f817c62dfc67ee80f068acb2966"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.312784 18047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "cb97544e596d4deba3f18046c96c0096"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "4c14c206fc082317973bed96606aef95"
server_key_iv: "1f398f817c62dfc67ee80f068acb2966"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.316104 18047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260620 11:40:19.338624 18064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.338969 18047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:40:19.339109 18047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "cb97544e596d4deba3f18046c96c0096"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "4c14c206fc082317973bed96606aef95"
server_key_iv: "1f398f817c62dfc67ee80f068acb2966"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.339227 18047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:19.352916 18047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:19.355571 18047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:19.355763 18047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:19.356065 18047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:19.356556 18047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:19.356614 18047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.356654 18047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:19.356676 18047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.363890 18047 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:44185
I20260620 11:40:19.364384 18047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:40:19.367286 18178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:44185 every 8 connection(s)
I20260620 11:40:19.368634 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18047
I20260620 11:40:19.368762 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:40:19.368959 5610 external_mini_cluster.cc:1428] Setting key 663ee82cd622093dbd11c7bc4a40c5bf
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
I20260620 11:40:19.382339 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:19.367194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:55131 (local address 127.5.122.190:35077)
0620 11:40:19.367260 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:19.367262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:19.367278 (+ 16us) server_negotiation.cc:407] Connection header received
0620 11:40:19.367916 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:19.367918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:19.367936 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:19.367987 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:19.368721 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.369444 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.371167 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.371504 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.371554 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:19.378531 (+ 6977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:19.378534 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:19.378535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:19.378546 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:19.380223 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:19.380861 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:19.380863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:19.380863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:19.380934 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:19.381272 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:19.381273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:19.381274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:19.381430 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:19.381477 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:19.382015 (+ 538us) server_negotiation.cc:299] Negotiation successful
0620 11:40:19.382170 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
I20260620 11:40:19.385226 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:19.366835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35077 (local address 127.5.122.130:55131)
0620 11:40:19.367045 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:19.367089 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:19.367773 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:19.368063 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:19.368067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:19.368276 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:19.368580 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:19.368598 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.369655 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.369656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:19.371055 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:19.371059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.371694 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.371695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:19.371864 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:19.372544 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:19.372557 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:19.378353 (+ 5796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:19.380408 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:19.380450 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:19.380454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:19.380766 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:19.381010 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:19.381011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:19.381011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:19.381184 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:19.381574 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:19.381583 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:19.381811 (+ 228us) client_negotiation.cc:769] Sending connection context
0620 11:40:19.384595 (+ 2784us) client_negotiation.cc:240] Negotiation successful
0620 11:40:19.384914 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":127,"threads_started":1}
I20260620 11:40:19.385797 18179 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35077
I20260620 11:40:19.385948 18179 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:19.386232 18179 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:19.386694 17829 ts_manager.cc:194] Registered new tserver with Master: cb97544e596d4deba3f18046c96c0096 (127.5.122.130:44185)
I20260620 11:40:19.387274 17829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:55131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:40:19.406044 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35077
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:19.529556 18184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:19.530025 18184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:19.530166 18184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:19.532243 18184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:19.532384 18184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:19.532478 18184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:40:19.535064 18184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.122.148:56157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:19.535403 18184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:19.535719 18184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:19.541841 18192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:19.541922 18190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:19.544227 18189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:19.543162 18184 server_base.cc:1034] running on GCE node
I20260620 11:40:19.544437 18184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:19.544749 18184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:19.545928 18184 hybrid_clock.cc:648] HybridClock initialized: now 1781955619545904 us; error 42 us; skew 500 ppm
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:19.548926 18184 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:40:19.549548 18184 webserver.cc:460] Webserver started at http://127.5.122.131:39225/ using document root <none> and password file <none>
I20260620 11:40:19.549839 18184 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:19.549911 18184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:19.550113 18184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:19.566092 18184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "31f780ddd7ec441f8779cc905e30051e"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "2aa5ebb9279f3640223d0abd8839f59c"
server_key_iv: "aa3e1673f175571c5e51d261880de7da"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.566601 18184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "31f780ddd7ec441f8779cc905e30051e"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "2aa5ebb9279f3640223d0abd8839f59c"
server_key_iv: "aa3e1673f175571c5e51d261880de7da"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.569319 18184 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260620 11:40:19.588141 18199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.594089 18184 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260620 11:40:19.594265 18184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "31f780ddd7ec441f8779cc905e30051e"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "2aa5ebb9279f3640223d0abd8839f59c"
server_key_iv: "aa3e1673f175571c5e51d261880de7da"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.594393 18184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:19.635393 18184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:19.638173 18184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:19.638352 18184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:19.638616 18184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:19.639045 18184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:19.639129 18184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.639171 18184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:19.639197 18184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.647280 18184 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:34121
I20260620 11:40:19.648239 18184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:40:19.648429 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18184
I20260620 11:40:19.648537 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:40:19.648766 5610 external_mini_cluster.cc:1428] Setting key 008fc1930db51c6a08172097a213dfb6
I20260620 11:40:19.648615 18312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:34121 every 8 connection(s)
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:19.662768 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:19.649835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:38901 (local address 127.5.122.190:35077)
0620 11:40:19.649880 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:19.649882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:19.650364 (+ 482us) server_negotiation.cc:407] Connection header received
0620 11:40:19.651114 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:19.651115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:19.651128 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:19.651160 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:19.652089 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.652883 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.654504 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.654854 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.654937 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:19.658937 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:19.658940 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:19.658941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:19.658950 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:19.660930 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:19.661526 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:19.661528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:19.661528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:19.661591 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:19.661899 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:19.661900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:19.661900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:19.662099 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:19.662160 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:19.662555 (+ 395us) server_negotiation.cc:299] Negotiation successful
0620 11:40:19.662647 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:40:19.663583 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:19.649508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35077 (local address 127.5.122.131:38901)
0620 11:40:19.650224 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:19.650278 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:19.650944 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:19.651366 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:19.651370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:19.651649 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:19.651955 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:19.651963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.653020 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.653021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:19.654394 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:19.654397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:19.654974 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:19.654975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:19.655145 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:19.655830 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:19.655842 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:19.658325 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:19.661050 (+ 2725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:19.661063 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:19.661066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:19.661431 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:19.661662 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:19.661662 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:19.661663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:19.661813 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:19.662196 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:19.662204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:19.662390 (+ 186us) client_negotiation.cc:769] Sending connection context
0620 11:40:19.663129 (+ 739us) client_negotiation.cc:240] Negotiation successful
0620 11:40:19.663323 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":108,"threads_started":1}
I20260620 11:40:19.664480 18313 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35077
I20260620 11:40:19.664598 18313 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:19.664812 18313 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:19.665231 17829 ts_manager.cc:194] Registered new tserver with Master: 31f780ddd7ec441f8779cc905e30051e (127.5.122.131:34121)
I20260620 11:40:19.665753 17829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:38901
I20260620 11:40:19.672376 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:40:19.673105 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 17910
I20260620 11:40:19.680358 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18047
I20260620 11:40:19.686354 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18184
I20260620 11:40:19.693075 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 17797
I20260620 11:40:19.700464 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35077
--webserver_interface=127.5.122.190
--webserver_port=36389
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:35077
--ranger_config_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:19.789098 18316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:19.789361 18316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:19.789387 18316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:19.790908 18316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:19.790975 18316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:19.790992 18316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:19.791024 18316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:19.791065 18316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:19.792907 18316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:35077
--ranger_config_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=36389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:19.793233 18316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:19.793524 18316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:19.799263 18321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:19.799292 18316 server_base.cc:1034] running on GCE node
W20260620 11:40:19.799256 18322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:19.799263 18324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:19.799800 18316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:19.800148 18316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:19.801323 18316 hybrid_clock.cc:648] HybridClock initialized: now 1781955619801287 us; error 47 us; skew 500 ppm
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:19.804440 18316 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:19.805001 18316 webserver.cc:460] Webserver started at http://127.5.122.190:36389/ using document root <none> and password file <none>
I20260620 11:40:19.805274 18316 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:19.805332 18316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 11:40:19 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955619, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:19.853700 18316 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:19.854676 18332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:19.854898 18316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:19.855032 18316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "d09670ae27a24184961918bf1e14a49b"
format_stamp: "Formatted at 2026-06-20 11:40:18 on dist-test-slave-szdr"
server_key: "d65e984cad4d858407f294bd5dc32009"
server_key_iv: "7ada3450743d9541dce207817c28248d"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:19.855568 18316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:19.865581 18316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:19.868008 18316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:19.868240 18316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:19.873348 18316 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:35077
I20260620 11:40:19.873386 18384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:35077 every 8 connection(s)
I20260620 11:40:19.873801 18316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:19.875319 18385 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:40:19.876255 18385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Bootstrap starting.
I20260620 11:40:19.877277 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18316
I20260620 11:40:19.877612 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:33677
--local_ip_for_outbound_sockets=127.5.122.129
--tserver_master_addrs=127.5.122.190:35077
--webserver_port=42341
--webserver_interface=127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:19.885144 18385 log.cc:824] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:19.887292 18385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:40:19.887646 18385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Bootstrap complete.
I20260620 11:40:19.890311 18385 raft_consensus.cc:348] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:19.890635 18385 raft_consensus.cc:729] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d09670ae27a24184961918bf1e14a49b, State: Initialized, Role: FOLLOWER
I20260620 11:40:19.890894 18385 consensus_queue.cc:260] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:19.890990 18385 raft_consensus.cc:388] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:40:19.891093 18385 raft_consensus.cc:482] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:40:19.891170 18385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 1 FOLLOWER]: Advancing to term 2
I20260620 11:40:19.892409 18385 raft_consensus.cc:504] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:19.892557 18385 leader_election.cc:302] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d09670ae27a24184961918bf1e14a49b; no voters:
I20260620 11:40:19.892817 18385 leader_election.cc:288] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 11:40:19.892858 18389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 2 FOLLOWER]: Leader election won for term 2
I20260620 11:40:19.892975 18389 raft_consensus.cc:686] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [term 2 LEADER]: Becoming Leader. State: Replica: d09670ae27a24184961918bf1e14a49b, State: Running, Role: LEADER
I20260620 11:40:19.893138 18385 sys_catalog.cc:564] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:19.893208 18389 consensus_queue.cc:237] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } }
I20260620 11:40:19.893501 18390 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d09670ae27a24184961918bf1e14a49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } } }
I20260620 11:40:19.893637 18390 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:19.893493 18389 sys_catalog.cc:455] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d09670ae27a24184961918bf1e14a49b. Latest consensus state: current_term: 2 leader_uuid: "d09670ae27a24184961918bf1e14a49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d09670ae27a24184961918bf1e14a49b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35077 } } }
I20260620 11:40:19.894135 18396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:40:19.894238 18389 sys_catalog.cc:458] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:19.894874 18396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:19.895077 18396 catalog_manager.cc:1259] Loaded cluster ID: c69bcf00ad42459b92bfe6a3286cc23b
I20260620 11:40:19.895147 18396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:40:19.895328 18385 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 11:40:19.896564 18396 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:40:19.896801 18396 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Loaded TSK: 0
I20260620 11:40:19.897055 18396 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 11:40:19.992274 18387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:19.992619 18387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:19.992700 18387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:19.994277 18387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:19.994354 18387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:19.994405 18387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:40:19.996650 18387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:33677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=42341
--tserver_master_addrs=127.5.122.190:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:19.996968 18387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:19.997262 18387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:20.002588 18425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:20.002738 18422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:20.002611 18423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:20.003468 18387 server_base.cc:1034] running on GCE node
I20260620 11:40:20.003867 18387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:20.004225 18387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:20.009069 18387 hybrid_clock.cc:648] HybridClock initialized: now 1781955620007824 us; error 1249 us; skew 500 ppm
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:20.013020 18387 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:40:20.013805 18387 webserver.cc:460] Webserver started at http://127.5.122.129:42341/ using document root <none> and password file <none>
I20260620 11:40:20.014206 18387 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:20.014374 18387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:20.031226 18387 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260620 11:40:20.032414 18433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.032629 18387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:40:20.032712 18387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "d215537622bf4c9ea7a2cb84d7bcc717"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "67051089d064575b41c2ef27bda9bd99"
server_key_iv: "74d9e15ab57c455196b771d837808448"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:20.033200 18387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:20.051290 18387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:20.054108 18387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:20.054291 18387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:20.054562 18387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:20.054920 18387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:20.054965 18387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.054996 18387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:20.055019 18387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.062629 18387 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:33677
I20260620 11:40:20.063197 18387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:40:20.063160 18546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:33677 every 8 connection(s)
I20260620 11:40:20.067939 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18387
I20260620 11:40:20.068295 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:44185
--local_ip_for_outbound_sockets=127.5.122.130
--tserver_master_addrs=127.5.122.190:35077
--webserver_port=43731
--webserver_interface=127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:20.079190 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:20.064648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:48679 (local address 127.5.122.190:35077)
0620 11:40:20.064885 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:20.064894 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:20.064937 (+ 43us) server_negotiation.cc:407] Connection header received
0620 11:40:20.065589 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:20.065597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:20.065663 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:20.065829 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:20.066536 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.067627 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.069039 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.069724 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.069847 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:20.074045 (+ 4198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:20.074053 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:20.074053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:20.074070 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:20.076534 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.077264 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.077269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.077269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.077402 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.077753 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.077754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.077754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.078193 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:20.078267 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:20.078743 (+ 476us) server_negotiation.cc:299] Negotiation successful
0620 11:40:20.078910 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
I20260620 11:40:20.080120 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:20.064586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35077 (local address 127.5.122.129:48679)
0620 11:40:20.064772 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:20.064817 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:20.065299 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:20.065866 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:20.065869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:20.066119 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:20.066409 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:20.066416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.067757 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.067758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:20.068923 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:20.068927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.069869 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.069871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:20.070052 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:20.070750 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:20.070762 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:20.073774 (+ 3012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:20.076685 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:20.076697 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:20.076700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:20.077067 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:20.077502 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:20.077503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:20.077504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:20.077664 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:20.078325 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:20.078334 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:20.078523 (+ 189us) client_negotiation.cc:769] Sending connection context
0620 11:40:20.079659 (+ 1136us) client_negotiation.cc:240] Negotiation successful
0620 11:40:20.079873 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260620 11:40:20.080964 18547 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35077
I20260620 11:40:20.081077 18547 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.081300 18547 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
W20260620 11:40:20.082422 18547 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:35077 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 11:40:20.082500 18547 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.082571 18547 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:20.083012 18547 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.083104 18547 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
W20260620 11:40:20.083386 18547 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 11:40:20.174408 18551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:20.174741 18551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:20.174780 18551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:20.176700 18551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:20.176765 18551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:20.176810 18551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:40:20.179339 18551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:44185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=43731
--tserver_master_addrs=127.5.122.190:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:20.179683 18551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:20.179972 18551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:20.185714 18557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:20.185739 18556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:20.185945 18551 server_base.cc:1034] running on GCE node
W20260620 11:40:20.186208 18559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:20.186484 18551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:20.186828 18551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:20.188026 18551 hybrid_clock.cc:648] HybridClock initialized: now 1781955620187855 us; error 188 us; skew 500 ppm
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:20.191547 18551 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:40:20.192184 18551 webserver.cc:460] Webserver started at http://127.5.122.130:43731/ using document root <none> and password file <none>
I20260620 11:40:20.192456 18551 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:20.192529 18551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:20.217285 18551 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260620 11:40:20.218395 18567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.218588 18551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:20.218680 18551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "cb97544e596d4deba3f18046c96c0096"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "4c14c206fc082317973bed96606aef95"
server_key_iv: "1f398f817c62dfc67ee80f068acb2966"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:20.219190 18551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:20.243350 18551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:20.245820 18551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:20.246064 18551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:20.246381 18551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:20.246791 18551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:20.246850 18551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.246886 18551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:20.246907 18551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.254833 18551 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:44185
I20260620 11:40:20.254913 18680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:44185 every 8 connection(s)
I20260620 11:40:20.255398 18551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:40:20.256281 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18551
I20260620 11:40:20.256614 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.122.148:56157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:34121
--local_ip_for_outbound_sockets=127.5.122.131
--tserver_master_addrs=127.5.122.190:35077
--webserver_port=39225
--webserver_interface=127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:20.268615 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:20.256777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:44243 (local address 127.5.122.190:35077)
0620 11:40:20.256823 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:20.256826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:20.257065 (+ 239us) server_negotiation.cc:407] Connection header received
0620 11:40:20.257773 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:20.257775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:20.257790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:20.257875 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:20.258777 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.259571 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.261203 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.261526 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.261580 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:20.264644 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:20.264646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:20.264647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:20.264657 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:20.266592 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.267234 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.267236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.267236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.267306 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.267670 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.267671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.267671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.267827 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:20.267877 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:20.268378 (+ 501us) server_negotiation.cc:299] Negotiation successful
0620 11:40:20.268495 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:40:20.269570 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:20.256459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35077 (local address 127.5.122.130:44243)
0620 11:40:20.256902 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:20.256951 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:20.257620 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:20.258062 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:20.258067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:20.258316 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:20.258637 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:20.258645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.259718 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.259719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:20.261074 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:20.261077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.261713 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.261715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:20.261874 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:20.262600 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:20.262612 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:20.264477 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:20.266720 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:20.266733 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:20.266736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:20.267126 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:20.267383 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:20.267384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:20.267384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:20.267558 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:20.267960 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:20.267970 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:20.268175 (+ 205us) client_negotiation.cc:769] Sending connection context
0620 11:40:20.269078 (+ 903us) client_negotiation.cc:240] Negotiation successful
0620 11:40:20.269288 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":86,"threads_started":1}
I20260620 11:40:20.270406 18681 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35077
I20260620 11:40:20.270519 18681 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.270759 18681 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
W20260620 11:40:20.271601 18681 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:35077 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 11:40:20.271687 18681 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.271759 18681 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:20.272159 18681 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.272239 18681 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
W20260620 11:40:20.272607 18681 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 11:40:20.357110 18684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:20.357352 18684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:20.357378 18684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:20.359077 18684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:20.359153 18684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:20.359197 18684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:40:20.361567 18684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.122.148:56157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:34121
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=39225
--tserver_master_addrs=127.5.122.190:35077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:20.361855 18684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:20.362215 18684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:20.369284 18690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:20.369313 18692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:20.369547 18689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:20.370815 18684 server_base.cc:1034] running on GCE node
I20260620 11:40:20.371168 18684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:20.371521 18684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:20.372686 18684 hybrid_clock.cc:648] HybridClock initialized: now 1781955620372668 us; error 31 us; skew 500 ppm
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:20.376654 18684 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:40:20.377442 18684 webserver.cc:460] Webserver started at http://127.5.122.131:39225/ using document root <none> and password file <none>
I20260620 11:40:20.377750 18684 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:20.377817 18684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:20.397117 18684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:20.398550 18700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.398942 18684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:20.399080 18684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "31f780ddd7ec441f8779cc905e30051e"
format_stamp: "Formatted at 2026-06-20 11:40:19 on dist-test-slave-szdr"
server_key: "2aa5ebb9279f3640223d0abd8839f59c"
server_key_iv: "aa3e1673f175571c5e51d261880de7da"
server_key_version: "kuduclusterkey@0"
I20260620 11:40:20.399698 18684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:20.423877 18684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:20.427021 18684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:20.427245 18684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:20.427536 18684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:20.427960 18684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:20.428026 18684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.428083 18684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:20.428109 18684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:20.436164 18684 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:34121
I20260620 11:40:20.436262 18814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:34121 every 8 connection(s)
I20260620 11:40:20.436689 18684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:40:20.444114 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18684
Jun 20 11:40:20 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955620, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:20.449703 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:20.437658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:43905 (local address 127.5.122.190:35077)
0620 11:40:20.437733 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:20.437736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:20.438009 (+ 273us) server_negotiation.cc:407] Connection header received
0620 11:40:20.438722 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:20.438725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:20.438745 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:20.438844 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:20.439592 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.440346 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.442007 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.442272 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.442366 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:20.445703 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:20.445705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:20.445705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:20.445715 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:20.447795 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.448410 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.448412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.448412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.448489 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.448904 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.448905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.448906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.449051 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:20.449123 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:20.449519 (+ 396us) server_negotiation.cc:299] Negotiation successful
0620 11:40:20.449627 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 11:40:20.450402 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:20.437669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35077 (local address 127.5.122.131:43905)
0620 11:40:20.437838 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:20.437880 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:20.438599 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:20.438916 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:20.438920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:20.439216 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:20.439474 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:20.439480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.440500 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.440501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:20.441850 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:20.441853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.442412 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.442414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:20.442600 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:20.443336 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:20.443349 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:20.445573 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:20.447907 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:20.447919 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:20.447923 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:20.448312 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:20.448582 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:20.448583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:20.448583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:20.448811 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:20.449110 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:20.449119 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:20.449327 (+ 208us) client_negotiation.cc:769] Sending connection context
0620 11:40:20.449895 (+ 568us) client_negotiation.cc:240] Negotiation successful
0620 11:40:20.450133 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260620 11:40:20.450887 18815 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35077
I20260620 11:40:20.451018 18815 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.451242 18815 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
W20260620 11:40:20.452009 18815 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:35077 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 11:40:20.452087 18815 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.452150 18815 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:20.452469 18815 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:20.452541 18815 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
W20260620 11:40:20.452834 18815 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 11:40:20.460088 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:20.445136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43874 (local address 127.5.122.190:35077)
0620 11:40:20.445269 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:20.445271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:20.445796 (+ 525us) server_negotiation.cc:407] Connection header received
0620 11:40:20.445886 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:20.445887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:20.445903 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:20.445997 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:20.448220 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.448980 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.450144 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:20.450822 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:20.451225 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:20.455801 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:20.455803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:20.455803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:20.455813 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:20.458119 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.459094 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.459096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.459097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.459174 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:20.459425 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:20.459426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:20.459426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:20.459592 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:20.459792 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:20.459830 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 11:40:20.459922 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
Jun 20 11:40:21 dist-test-slave-szdr krb5kdc[17353](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781955621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:21.083606 18547 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:21.083788 18547 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:21.084175 18349 ts_manager.cc:194] Registered new tserver with Master: d215537622bf4c9ea7a2cb84d7bcc717 (127.5.122.129:33677)
I20260620 11:40:21.084885 18349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:48679
I20260620 11:40:21.272833 18681 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:21.273022 18681 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:21.273428 18349 ts_manager.cc:194] Registered new tserver with Master: cb97544e596d4deba3f18046c96c0096 (127.5.122.130:44185)
I20260620 11:40:21.274014 18349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:44243
I20260620 11:40:21.453049 18815 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:21.453225 18815 heartbeater.cc:511] Master 127.5.122.190:35077 requested a full tablet report, sending...
I20260620 11:40:21.453644 18349 ts_manager.cc:194] Registered new tserver with Master: 31f780ddd7ec441f8779cc905e30051e (127.5.122.131:34121)
I20260620 11:40:21.454201 18349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:43905
I20260620 11:40:21.459106 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:40:21.471484 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.460242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.5.122.190:35077)
0620 11:40:21.460573 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:21.460575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:21.460753 (+ 178us) server_negotiation.cc:407] Connection header received
0620 11:40:21.460781 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:21.460782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:21.460795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:21.460858 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:21.461642 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.462467 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.463547 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.463835 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.463908 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.467819 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:21.467821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:21.467822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:21.467831 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:21.469669 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:21.470127 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:21.470129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:21.470129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:21.470205 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:21.470467 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:21.470468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:21.470477 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:21.470640 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:21.470725 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:21.471266 (+ 541us) server_negotiation.cc:299] Negotiation successful
0620 11:40:21.471360 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":68,"threads_started":1}
I20260620 11:40:21.473767 18349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:40:21.476410 18349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Jun 20 11:40:21 dist-test-slave-szdr krb5kdc[17353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955621, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for HTTP/127.5.122.148@KRBTEST.COM
I20260620 11:40:21.501279 18842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.490775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:44185 (local address 127.0.0.1:60098)
0620 11:40:21.492396 (+ 1621us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:21.492404 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:21.492436 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:21.492826 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:21.492830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:21.492832 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:21.492917 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.492919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.495288 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.495289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.497077 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.497080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.500920 (+ 3840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.500921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.501030 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.501050 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 11:40:21.501127 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 11:40:21.501187 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1609,"thread_start_us":76,"threads_started":1}
I20260620 11:40:21.502835 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.490886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.5.122.130:44185)
0620 11:40:21.492554 (+ 1668us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:21.492561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:21.492572 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:40:21.492613 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:21.492615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:21.492633 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:21.492713 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:21.493004 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.495187 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.499995 (+ 4808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.500810 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.500920 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.500996 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:21.502688 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0620 11:40:21.502759 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1631,"thread_start_us":103,"threads_started":1}
I20260620 11:40:21.502895 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.490307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42934 (local address 127.5.122.131:34121)
0620 11:40:21.494397 (+ 4090us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:21.494403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:21.494416 (+ 13us) server_negotiation.cc:407] Connection header received
0620 11:40:21.494457 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:21.494459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:21.494489 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:21.494579 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:21.497197 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.498317 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.501040 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.501808 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.501893 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.501993 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:21.502757 (+ 764us) server_negotiation.cc:299] Negotiation successful
0620 11:40:21.502807 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4043,"thread_start_us":80,"threads_started":1}
I20260620 11:40:21.502674 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.490231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:34121 (local address 127.0.0.1:42934)
0620 11:40:21.492051 (+ 1820us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:21.492105 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:21.492178 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:21.494687 (+ 2509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:21.494688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:21.494690 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:21.494741 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.494743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.498420 (+ 3677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.498421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.499869 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.499873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.502033 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.502034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.502446 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.502451 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:40:21.502515 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 11:40:21.502564 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1781,"thread_start_us":120,"threads_started":1}
I20260620 11:40:21.504093 18614 tablet_service.cc:1467] Processing CreateTablet for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 (DEFAULT_TABLE table=test-table [id=8bf5776f5e0a4bd6985a9212fa4a175a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:21.504247 18749 tablet_service.cc:1467] Processing CreateTablet for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 (DEFAULT_TABLE table=test-table [id=8bf5776f5e0a4bd6985a9212fa4a175a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:21.504518 18614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:21.504649 18749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:21.508492 18848 tablet_bootstrap.cc:492] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Bootstrap starting.
I20260620 11:40:21.509910 18848 tablet_bootstrap.cc:654] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:21.510396 18848 log.cc:824] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:21.514557 18849 tablet_bootstrap.cc:492] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Bootstrap starting.
I20260620 11:40:21.515362 18849 tablet_bootstrap.cc:654] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:21.515715 18849 log.cc:824] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:21.515776 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.490566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:33677 (local address 127.0.0.1:44562)
0620 11:40:21.492282 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:21.492290 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:21.492324 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:21.511363 (+ 19039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:21.511365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:21.511369 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:21.511441 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.511444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.513254 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.513255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.514466 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.514480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.515519 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.515520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.515594 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.515601 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 11:40:21.515634 (+ 33us) client_negotiation.cc:240] Negotiation successful
0620 11:40:21.515688 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1701,"thread_start_us":86,"threads_started":1}
I20260620 11:40:21.516252 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.491236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.5.122.129:33677)
0620 11:40:21.511145 (+ 19909us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:21.511151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:21.511167 (+ 16us) server_negotiation.cc:407] Connection header received
0620 11:40:21.511209 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:21.511212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:21.511230 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:21.511541 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:21.511636 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.513094 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.514595 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.515382 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.515995 (+ 613us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.516082 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:21.516115 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 11:40:21.516165 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19860,"thread_start_us":62,"threads_started":1}
I20260620 11:40:21.517025 18468 tablet_service.cc:1467] Processing CreateTablet for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 (DEFAULT_TABLE table=test-table [id=8bf5776f5e0a4bd6985a9212fa4a175a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:21.517436 18468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:21.522315 18852 tablet_bootstrap.cc:492] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717: Bootstrap starting.
I20260620 11:40:21.522406 18849 tablet_bootstrap.cc:492] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: No bootstrap required, opened a new log
I20260620 11:40:21.522493 18849 ts_tablet_manager.cc:1397] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260620 11:40:21.523144 18852 tablet_bootstrap.cc:654] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:21.523557 18852 log.cc:824] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:21.525362 18849 raft_consensus.cc:348] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.525573 18849 raft_consensus.cc:374] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:21.525617 18849 raft_consensus.cc:729] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f780ddd7ec441f8779cc905e30051e, State: Initialized, Role: FOLLOWER
I20260620 11:40:21.525771 18849 consensus_queue.cc:260] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.526208 18849 ts_tablet_manager.cc:1428] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260620 11:40:21.526382 18815 heartbeater.cc:503] Master 127.5.122.190:35077 was elected leader, sending a full tablet report...
I20260620 11:40:21.527262 18848 tablet_bootstrap.cc:492] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: No bootstrap required, opened a new log
I20260620 11:40:21.527354 18848 ts_tablet_manager.cc:1397] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20260620 11:40:21.528570 18852 tablet_bootstrap.cc:492] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717: No bootstrap required, opened a new log
I20260620 11:40:21.528659 18852 ts_tablet_manager.cc:1397] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260620 11:40:21.530349 18848 raft_consensus.cc:348] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.530565 18848 raft_consensus.cc:374] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:21.530614 18848 raft_consensus.cc:729] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb97544e596d4deba3f18046c96c0096, State: Initialized, Role: FOLLOWER
I20260620 11:40:21.530776 18848 consensus_queue.cc:260] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.531114 18848 ts_tablet_manager.cc:1428] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260620 11:40:21.531271 18681 heartbeater.cc:503] Master 127.5.122.190:35077 was elected leader, sending a full tablet report...
I20260620 11:40:21.532356 18852 raft_consensus.cc:348] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.532552 18852 raft_consensus.cc:374] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:21.532591 18852 raft_consensus.cc:729] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d215537622bf4c9ea7a2cb84d7bcc717, State: Initialized, Role: FOLLOWER
I20260620 11:40:21.532733 18852 consensus_queue.cc:260] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.533013 18852 ts_tablet_manager.cc:1428] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 11:40:21.533124 18547 heartbeater.cc:503] Master 127.5.122.190:35077 was elected leader, sending a full tablet report...
W20260620 11:40:21.565070 18548 tablet.cc:2369] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:40:21.630491 18854 raft_consensus.cc:482] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:21.630681 18854 raft_consensus.cc:504] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.631196 18854 leader_election.cc:288] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d215537622bf4c9ea7a2cb84d7bcc717 (127.5.122.129:33677), cb97544e596d4deba3f18046c96c0096 (127.5.122.130:44185)
I20260620 11:40:21.635442 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.631647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:47791 (local address 127.5.122.130:44185)
0620 11:40:21.631711 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:21.631714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:21.631874 (+ 160us) server_negotiation.cc:407] Connection header received
0620 11:40:21.631977 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:21.631979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:21.631989 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:21.632017 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:21.632265 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.633001 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.634498 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.635046 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.635099 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.635108 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:21.635324 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 11:40:21.635367 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 11:40:21.635519 18847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.631698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:50237 (local address 127.5.122.129:33677)
0620 11:40:21.631744 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:21.631746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:21.631873 (+ 127us) server_negotiation.cc:407] Connection header received
0620 11:40:21.631981 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:21.631982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:21.631991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:21.632022 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:21.632288 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.633062 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.634522 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.635085 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.635138 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.635148 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:21.635377 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 11:40:21.635426 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:40:21.635811 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.631461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:33677 (local address 127.5.122.131:50237)
0620 11:40:21.631804 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:21.631815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:21.631902 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:21.632093 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:21.632093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:21.632096 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:21.632189 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.632192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.633165 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.633166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.634390 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.634392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.635209 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.635209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.635291 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.635294 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:40:21.635661 (+ 367us) client_negotiation.cc:240] Negotiation successful
0620 11:40:21.635712 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":5,"thread_start_us":96,"threads_started":1}
I20260620 11:40:21.636112 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.631411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:44185 (local address 127.5.122.131:47791)
0620 11:40:21.631804 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:21.631816 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:21.631903 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:21.632088 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:21.632089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:21.632091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:21.632197 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.632198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.633102 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.633103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.634387 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:21.634390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.635186 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.635187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:21.635250 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.635254 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:40:21.635967 (+ 713us) client_negotiation.cc:240] Negotiation successful
0620 11:40:21.636018 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":8,"thread_start_us":111,"threads_started":1}
I20260620 11:40:21.636226 18483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ac8ed56e7e4d9fa06598b45fe6efc6" candidate_uuid: "31f780ddd7ec441f8779cc905e30051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" is_pre_election: true
I20260620 11:40:21.636425 18483 raft_consensus.cc:2454] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31f780ddd7ec441f8779cc905e30051e in term 0.
I20260620 11:40:21.636454 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ac8ed56e7e4d9fa06598b45fe6efc6" candidate_uuid: "31f780ddd7ec441f8779cc905e30051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb97544e596d4deba3f18046c96c0096" is_pre_election: true
I20260620 11:40:21.636624 18635 raft_consensus.cc:2454] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31f780ddd7ec441f8779cc905e30051e in term 0.
I20260620 11:40:21.636706 18704 leader_election.cc:302] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f780ddd7ec441f8779cc905e30051e, d215537622bf4c9ea7a2cb84d7bcc717; no voters:
I20260620 11:40:21.636878 18854 raft_consensus.cc:2781] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:40:21.636935 18854 raft_consensus.cc:482] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:40:21.636956 18854 raft_consensus.cc:3037] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:21.637949 18854 raft_consensus.cc:504] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.638150 18854 leader_election.cc:288] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [CANDIDATE]: Term 1 election: Requested vote from peers d215537622bf4c9ea7a2cb84d7bcc717 (127.5.122.129:33677), cb97544e596d4deba3f18046c96c0096 (127.5.122.130:44185)
I20260620 11:40:21.638370 18483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ac8ed56e7e4d9fa06598b45fe6efc6" candidate_uuid: "31f780ddd7ec441f8779cc905e30051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d215537622bf4c9ea7a2cb84d7bcc717"
I20260620 11:40:21.638362 18635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96ac8ed56e7e4d9fa06598b45fe6efc6" candidate_uuid: "31f780ddd7ec441f8779cc905e30051e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb97544e596d4deba3f18046c96c0096"
I20260620 11:40:21.638484 18483 raft_consensus.cc:3037] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:21.638535 18635 raft_consensus.cc:3037] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:21.639334 18483 raft_consensus.cc:2454] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31f780ddd7ec441f8779cc905e30051e in term 1.
I20260620 11:40:21.639556 18704 leader_election.cc:302] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f780ddd7ec441f8779cc905e30051e, d215537622bf4c9ea7a2cb84d7bcc717; no voters:
I20260620 11:40:21.639750 18635 raft_consensus.cc:2454] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31f780ddd7ec441f8779cc905e30051e in term 1.
I20260620 11:40:21.639755 18854 raft_consensus.cc:2781] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:21.639899 18854 raft_consensus.cc:686] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 1 LEADER]: Becoming Leader. State: Replica: 31f780ddd7ec441f8779cc905e30051e, State: Running, Role: LEADER
I20260620 11:40:21.639988 18854 consensus_queue.cc:237] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } }
I20260620 11:40:21.640897 18349 catalog_manager.cc:5495] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e reported cstate change: term changed from 0 to 1, leader changed from <none> to 31f780ddd7ec441f8779cc905e30051e (127.5.122.131). New cstate: current_term: 1 leader_uuid: "31f780ddd7ec441f8779cc905e30051e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31f780ddd7ec441f8779cc905e30051e" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 } health_report { overall_health: UNKNOWN } } }
I20260620 11:40:21.680028 18843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.676716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.5.122.131:34121)
0620 11:40:21.676788 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:21.676790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:21.676801 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:40:21.676843 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:21.676844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:21.676866 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:21.676950 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:40:21.677115 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.677874 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.678990 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:21.679271 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:21.679353 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:21.679552 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:40:21.679767 (+ 215us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:40:21.679868 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:21.679921 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 11:40:21.679947 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 11:40:21.684478 18483 raft_consensus.cc:1261] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P d215537622bf4c9ea7a2cb84d7bcc717 [term 1 FOLLOWER]: Refusing update from remote peer 31f780ddd7ec441f8779cc905e30051e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:21.684913 18854 consensus_queue.cc:1035] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d215537622bf4c9ea7a2cb84d7bcc717" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 33677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:21.686435 18635 raft_consensus.cc:1261] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 1 FOLLOWER]: Refusing update from remote peer 31f780ddd7ec441f8779cc905e30051e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:21.686738 18854 consensus_queue.cc:1035] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb97544e596d4deba3f18046c96c0096" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 11:40:21.687943 18816 tablet.cc:2369] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:40:21.690014 18862 mvcc.cc:205] Tried to move back new op lower bound from 7298890226415964160 to 7298890226238197760. Current Snapshot: MvccSnapshot[applied={T|T < 7298890226415964160}]
I20260620 11:40:21.692667 18349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:21.692775 18349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:21.693188 18865 mvcc.cc:205] Tried to move back new op lower bound from 7298890226415964160 to 7298890226238197760. Current Snapshot: MvccSnapshot[applied={T|T < 7298890226415964160}]
I20260620 11:40:21.693729 18349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d09670ae27a24184961918bf1e14a49b: Sending DeleteTablet for 3 replicas of tablet 96ac8ed56e7e4d9fa06598b45fe6efc6
I20260620 11:40:21.694123 18749 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:21 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:42934
I20260620 11:40:21.694392 18614 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:21 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:60098
I20260620 11:40:21.694394 18468 tablet_service.cc:1514] Processing DeleteTablet for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:21 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:44562
I20260620 11:40:21.694825 18873 tablet_replica.cc:330] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: stopping tablet replica
I20260620 11:40:21.694943 18873 raft_consensus.cc:2229] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:40:21.695013 18873 raft_consensus.cc:2258] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:21.695273 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18387
I20260620 11:40:21.698127 18872 tablet_replica.cc:330] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: stopping tablet replica
I20260620 11:40:21.698257 18872 raft_consensus.cc:2229] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:40:21.698356 18872 raft_consensus.cc:2258] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:21.699177 18872 ts_tablet_manager.cc:1905] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:21.701699 18872 ts_tablet_manager.cc:1918] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:21.701860 18872 log.cc:1196] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/96ac8ed56e7e4d9fa06598b45fe6efc6
I20260620 11:40:21.702205 18873 ts_tablet_manager.cc:1905] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:21.702355 18872 ts_tablet_manager.cc:1939] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P 31f780ddd7ec441f8779cc905e30051e: Deleting consensus metadata
I20260620 11:40:21.702889 18335 catalog_manager.cc:4841] TS 31f780ddd7ec441f8779cc905e30051e (127.5.122.131:34121): tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 (table test-table [id=8bf5776f5e0a4bd6985a9212fa4a175a]) successfully deleted
I20260620 11:40:21.703960 18873 ts_tablet_manager.cc:1918] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:21.704043 18873 log.cc:1196] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/96ac8ed56e7e4d9fa06598b45fe6efc6
I20260620 11:40:21.704329 18873 ts_tablet_manager.cc:1939] T 96ac8ed56e7e4d9fa06598b45fe6efc6 P cb97544e596d4deba3f18046c96c0096: Deleting consensus metadata
I20260620 11:40:21.704756 18334 catalog_manager.cc:4841] TS cb97544e596d4deba3f18046c96c0096 (127.5.122.130:44185): tablet 96ac8ed56e7e4d9fa06598b45fe6efc6 (table test-table [id=8bf5776f5e0a4bd6985a9212fa4a175a]) successfully deleted
W20260620 11:40:21.707713 18336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:33677 (error 108)
I20260620 11:40:21.708691 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:21.708553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:33677 (local address 127.0.0.1:44570)
0620 11:40:21.708608 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:21.708639 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 11:40:21.708925 18336 catalog_manager.cc:4584] TS d215537622bf4c9ea7a2cb84d7bcc717 (127.5.122.129:33677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96ac8ed56e7e4d9fa06598b45fe6efc6: Network error: Client connection negotiation failed: client connection to 127.5.122.129:33677: connect: Connection refused (error 111)
I20260620 11:40:21.709663 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18551
I20260620 11:40:21.715004 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18684
I20260620 11:40:21.722322 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18316
I20260620 11:40:21.728173 5610 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 11:40:22.073823 5610 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 11:40:22.073937 5610 mini_ranger.cc:66] Stopping Ranger...
I20260620 11:40:22.419849 5610 mini_ranger.cc:68] Stopped Ranger
2026-06-20 11:40:22.420 UTC [17383] LOG: received smart shutdown request
2026-06-20 11:40:22.424 UTC [17383] LOG: background worker "logical replication launcher" (PID 17392) exited with exit code 1
2026-06-20 11:40:22.424 UTC [17387] LOG: shutting down
2026-06-20 11:40:22.607 UTC [17383] LOG: database system is shut down
2026-06-20T11:40:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50926 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:40:22 dist-test-slave-szdr krb5kdc[18884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:40:22 dist-test-slave-szdr krb5kdc[18884](info): set up 2 sockets
Jun 20 11:40:22 dist-test-slave-szdr krb5kdc[18884](info): commencing operation
krb5kdc: starting...
W20260620 11:40:23.782960 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:40:23 dist-test-slave-szdr krb5kdc[18884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:40:23.910122 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37871
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:37871,127.5.122.189:44495,127.5.122.188:45707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:23.997781 18897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:23.998059 18897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:23.998086 18897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:23.999455 18897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:23.999518 18897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:23.999541 18897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:23.999555 18897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:23.999567 18897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:24.001384 18897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:37871,127.5.122.189:44495,127.5.122.188:45707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:24.001730 18897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:24.002048 18897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:24.007565 18905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:24.007688 18903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:24.007688 18902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:24.008121 18897 server_base.cc:1034] running on GCE node
I20260620 11:40:24.008437 18897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:24.008764 18897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:24.009943 18897 hybrid_clock.cc:648] HybridClock initialized: now 1781955624009906 us; error 56 us; skew 500 ppm
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:24.012773 18897 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:24.013305 18897 webserver.cc:460] Webserver started at http://127.5.122.190:37723/ using document root <none> and password file <none>
I20260620 11:40:24.013564 18897 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:24.013636 18897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:24.013793 18897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:24.015050 18897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "ab5c1eee06524baabb91c9785f74600a"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "0eeaeb61b03bbe995bd2732f6575faff"
server_key_iv: "f38fe5967e52bff016e24b54dbe7931f"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.015440 18897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "ab5c1eee06524baabb91c9785f74600a"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "0eeaeb61b03bbe995bd2732f6575faff"
server_key_iv: "f38fe5967e52bff016e24b54dbe7931f"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.016907 18897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:40:24.017714 18912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:24.017904 18897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:24.018069 18897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "ab5c1eee06524baabb91c9785f74600a"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "0eeaeb61b03bbe995bd2732f6575faff"
server_key_iv: "f38fe5967e52bff016e24b54dbe7931f"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.018162 18897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:24.055508 18897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:24.057900 18897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:24.058197 18897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:24.063001 18897 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:37871
I20260620 11:40:24.063074 18964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:37871 every 8 connection(s)
I20260620 11:40:24.063481 18897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:24.064419 18965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:24.066205 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18897
I20260620 11:40:24.066334 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:40:24.066545 5610 external_mini_cluster.cc:1428] Setting key 24c0c14b9a1194b371f859054f5fd0d5
I20260620 11:40:24.066305 18965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.189@KRBTEST.COM" created.
I20260620 11:40:24.078006 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.067181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37871 (local address 127.0.0.1:46722)
0620 11:40:24.067378 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.067425 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.067971 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.068308 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.068310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.068615 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.068781 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.068786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.069842 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.069844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.070647 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.070650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.071243 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.071244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.071311 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.071928 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.071937 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.073621 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.076075 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.076083 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.076085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.076330 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.076636 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.076637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.076637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.076742 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.077227 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.077233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.077306 (+ 73us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.077439 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.077722 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":136,"thread_start_us":96,"threads_started":1}
I20260620 11:40:24.078003 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.067219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46722 (local address 127.5.122.190:37871)
0620 11:40:24.067419 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.067431 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.067576 (+ 145us) server_negotiation.cc:407] Connection header received
0620 11:40:24.068166 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.068169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.068192 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.068299 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.068891 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.069733 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.070745 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.071136 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.071262 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.073720 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.073727 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.073727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.073734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.075832 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.076444 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.076448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.076448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.076556 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.076817 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.076819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.076819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.077146 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.077254 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.077462 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.077669 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
I20260620 11:40:24.078881 18965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:40:24.079339 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.079178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40202)
0620 11:40:24.079237 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.079299 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 11:40:24.079519 18913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
I20260620 11:40:24.080044 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.079919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40214)
0620 11:40:24.079971 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.080002 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 11:40:24.080181 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:44495: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab.
Entry for principal kudu/127.5.122.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab.
I20260620 11:40:24.094482 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:44495
--webserver_interface=127.5.122.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab
--principal=kudu/127.5.122.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:37871,127.5.122.189:44495,127.5.122.188:45707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:24.129547 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } attempt: 1
I20260620 11:40:24.130290 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.130104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40218)
0620 11:40:24.130187 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.130242 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 11:40:24.130745 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.130645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40232)
0620 11:40:24.130692 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.130726 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 11:40:24.130944 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:44495: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
W20260620 11:40:24.181203 18972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:24.181459 18972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:24.181485 18972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:24.182777 18972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:24.182838 18972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:24.182853 18972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:24.182866 18972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:24.182878 18972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:24.184542 18972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:37871,127.5.122.189:44495,127.5.122.188:45707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab
--principal=kudu/127.5.122.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:44495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:24.184836 18972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:24.185104 18972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:24.190524 18980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:24.190500 18977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:24.190490 18978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:24.190618 18972 server_base.cc:1034] running on GCE node
I20260620 11:40:24.191090 18972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:24.191461 18972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:24.192685 18972 hybrid_clock.cc:648] HybridClock initialized: now 1781955624192663 us; error 34 us; skew 500 ppm
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:24.195613 18972 init.cc:377] Logged in from keytab as kudu/127.5.122.189@KRBTEST.COM (short username kudu)
I20260620 11:40:24.196188 18972 webserver.cc:460] Webserver started at http://127.5.122.189:37143/ using document root <none> and password file <none>
I20260620 11:40:24.196444 18972 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:24.196517 18972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:24.196673 18972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:24.197793 18972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data/instance:
uuid: "bbd0704fc5f141f78054feb9bad2a8bb"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "f7d76baa64c832a3d854b3dccc792233"
server_key_iv: "42d526d52b563e492629cedf85e4c8dd"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.198208 18972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/wal/instance:
uuid: "bbd0704fc5f141f78054feb9bad2a8bb"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "f7d76baa64c832a3d854b3dccc792233"
server_key_iv: "42d526d52b563e492629cedf85e4c8dd"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.199132 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } attempt: 2
I20260620 11:40:24.199575 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.199432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40234)
0620 11:40:24.199488 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.199533 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 11:40:24.199740 18972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:24.200001 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.199854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40238)
0620 11:40:24.199957 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.199983 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260620 11:40:24.200233 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:44495: Network error: Client connection negotiation failed: client connection to 127.5.122.189:44495: connect: Connection refused (error 111)
I20260620 11:40:24.200728 18987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:24.200909 18972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:24.201007 18972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/wal
uuid: "bbd0704fc5f141f78054feb9bad2a8bb"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "f7d76baa64c832a3d854b3dccc792233"
server_key_iv: "42d526d52b563e492629cedf85e4c8dd"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.201125 18972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:24.215790 18972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:24.218278 18972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:24.218456 18972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:24.223155 18972 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.189:44495
I20260620 11:40:24.223548 19039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.189:44495 every 8 connection(s)
I20260620 11:40:24.223598 18972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
I20260620 11:40:24.224429 19040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:24.226177 19040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:40:24.230269 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 18972
I20260620 11:40:24.230412 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-1/wal/instance
I20260620 11:40:24.230644 5610 external_mini_cluster.cc:1428] Setting key ddfd41804ee21889f27e99f6e6530819
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:24.236362 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.227150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.5.122.190:37871)
0620 11:40:24.227223 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.227225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.227303 (+ 78us) server_negotiation.cc:407] Connection header received
0620 11:40:24.227776 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.227776 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.227786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.227850 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.228515 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.229029 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.230063 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.230289 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.230394 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.233129 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.233130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.233131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.233146 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.234796 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.235278 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.235280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.235281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.235353 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.235641 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.235641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.235642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.235769 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.235847 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.236166 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.236263 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:40:24.236593 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.226957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37871 (local address 127.0.0.1:46732)
0620 11:40:24.227162 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.227214 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.227688 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.227913 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.227917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.228194 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.228409 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.228414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.229123 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.229123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.229945 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.229947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.230424 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.230425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.230599 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.231193 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.231204 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.232983 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.234892 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.234902 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.234904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.235190 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.235429 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.235430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.235431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.235564 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.235838 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.235847 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.236018 (+ 171us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.236146 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.236329 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":67,"threads_started":1}
I20260620 11:40:24.237299 19040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.122.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.188@KRBTEST.COM" created.
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:40:24.246069 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.237701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.5.122.189:44495)
0620 11:40:24.237883 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.237886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.237903 (+ 17us) server_negotiation.cc:407] Connection header received
0620 11:40:24.237920 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.237921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.237932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.238044 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.238330 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.239091 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.239969 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.240290 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.240382 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.242551 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.242560 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.242560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.242568 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.244594 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.245053 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.245056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.245056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.245121 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.245346 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.245346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.245347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.245535 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.245608 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.245784 (+ 176us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.245865 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260620 11:40:24.246177 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.237614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40248)
0620 11:40:24.237672 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.237683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.237726 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.238043 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.238044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.238205 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.238252 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.238254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.239184 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.239185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.239877 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.239880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.240380 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.240381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.240449 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.240922 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.240928 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.242463 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.244693 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.244695 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.244695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.244934 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.245206 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.245206 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.245207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.245280 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.245610 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.245611 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.245667 (+ 56us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.245757 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.245995 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":73}
I20260620 11:40:24.246960 19040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:40:24.247439 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.247279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57140)
0620 11:40:24.247343 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.247403 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 11:40:24.247646 18988 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
I20260620 11:40:24.248169 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.248051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57146)
0620 11:40:24.248104 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.248137 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 11:40:24.248337 19040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:45707: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab.
Entry for principal kudu/127.5.122.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab.
I20260620 11:40:24.262203 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:45707
--webserver_interface=127.5.122.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab
--principal=kudu/127.5.122.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:37871,127.5.122.189:44495,127.5.122.188:45707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:24.291460 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } attempt: 3
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:40:24.297592 19040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } attempt: 1
I20260620 11:40:24.298146 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.297992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57162)
0620 11:40:24.298055 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.298091 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 11:40:24.298483 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.298375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57178)
0620 11:40:24.298427 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.298455 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 11:40:24.298863 19040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:45707: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
I20260620 11:40:24.300633 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.292129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.5.122.189:44495)
0620 11:40:24.292223 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.292225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.292239 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:40:24.292281 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.292282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.292303 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.292373 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.292807 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.293341 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.294191 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.294441 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.294548 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.297203 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.297205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.297205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.297221 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.299385 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.299844 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.299845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.299846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.299910 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.300136 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.300137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.300137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.300267 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.300321 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.300500 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.300577 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 11:40:24.300694 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.292042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40258)
0620 11:40:24.292100 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.292114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.292214 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.292382 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.292382 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.292615 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.292699 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.292702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.293452 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.293453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.294096 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.294099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.294552 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.294553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.294652 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.295279 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.295286 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.297085 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.299501 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.299503 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.299503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.299749 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.299995 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.299996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.299996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.300068 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.300336 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.300337 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.300396 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.300473 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.300597 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 11:40:24.301141 18965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:40:24.301503 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.301389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57194)
0620 11:40:24.301440 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.301476 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 11:40:24.302008 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.301785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57206)
0620 11:40:24.301858 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.301891 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260620 11:40:24.302126 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:45707: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
I20260620 11:40:24.333287 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } attempt: 1
I20260620 11:40:24.333884 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.333729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57210)
0620 11:40:24.333805 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.333851 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 11:40:24.334296 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.334186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57226)
0620 11:40:24.334240 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.334275 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 11:40:24.334400 18965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:45707: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
W20260620 11:40:24.353344 19047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:24.353621 19047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:24.353648 19047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:24.355051 19047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:24.355119 19047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:24.355134 19047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:24.355147 19047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:24.355160 19047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:24.356961 19047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:37871,127.5.122.189:44495,127.5.122.188:45707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab
--principal=kudu/127.5.122.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:45707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:24.357290 19047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:24.357564 19047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:24.363003 19052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:24.362998 19055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:24.363035 19053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:24.363389 19047 server_base.cc:1034] running on GCE node
I20260620 11:40:24.363667 19047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:24.364058 19047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:24.365244 19047 hybrid_clock.cc:648] HybridClock initialized: now 1781955624365217 us; error 38 us; skew 500 ppm
I20260620 11:40:24.367049 19040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } attempt: 2
I20260620 11:40:24.367522 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.367392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57238)
0620 11:40:24.367459 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.367484 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:24.367897 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.367768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57246)
0620 11:40:24.367830 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.367867 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 11:40:24.368098 19040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:45707: Network error: Client connection negotiation failed: client connection to 127.5.122.188:45707: connect: Connection refused (error 111)
I20260620 11:40:24.368319 19047 init.cc:377] Logged in from keytab as kudu/127.5.122.188@KRBTEST.COM (short username kudu)
I20260620 11:40:24.368836 19047 webserver.cc:460] Webserver started at http://127.5.122.188:45419/ using document root <none> and password file <none>
I20260620 11:40:24.369086 19047 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:24.369138 19047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:24.369283 19047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:24.370517 19047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data/instance:
uuid: "9df21a28d5154d89bd357903fc1b9288"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "cbaeaa9011a9b313b880cd89cb395e7e"
server_key_iv: "70eabdbd38df709910f88240fc5d3ef3"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.370919 19047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/wal/instance:
uuid: "9df21a28d5154d89bd357903fc1b9288"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "cbaeaa9011a9b313b880cd89cb395e7e"
server_key_iv: "70eabdbd38df709910f88240fc5d3ef3"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.372377 19047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:40:24.373188 19062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:24.373370 19047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:40:24.373481 19047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/wal
uuid: "9df21a28d5154d89bd357903fc1b9288"
format_stamp: "Formatted at 2026-06-20 11:40:24 on dist-test-slave-szdr"
server_key: "cbaeaa9011a9b313b880cd89cb395e7e"
server_key_iv: "70eabdbd38df709910f88240fc5d3ef3"
server_key_version: "encryptionkey@0"
I20260620 11:40:24.373564 19047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:24.396833 19047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:24.399680 19047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:24.399818 19047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:24.404532 19047 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.188:45707
I20260620 11:40:24.404556 19114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.188:45707 every 8 connection(s)
I20260620 11:40:24.404983 19047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
I20260620 11:40:24.406045 19115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:24.407773 19115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:40:24.408196 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19047
I20260620 11:40:24.408313 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781955466149030-5610-0/minicluster-data/master-2/wal/instance
I20260620 11:40:24.408493 5610 external_mini_cluster.cc:1428] Setting key e18480ba3b83993992aae7a3e1137454
I20260620 11:40:24.409581 18965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } attempt: 2
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:40:24.419423 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.410012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57260)
0620 11:40:24.410054 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.410066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.410116 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.410487 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.410488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.410703 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.410748 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.410750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.411832 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.411833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.412601 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.412603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.413059 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.413060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.413116 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.413776 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.413781 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.415372 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.417810 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.417812 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.417812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.418111 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.418418 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.418419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.418419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.418499 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.418979 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.418981 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.419041 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.419193 (+ 152us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.419306 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260620 11:40:24.419907 19120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.410172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.5.122.188:45707)
0620 11:40:24.410352 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.410354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.410361 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:40:24.410378 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.410378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.410393 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.410478 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.410844 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.411706 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.412684 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.412966 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.413318 (+ 352us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.415559 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.415569 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.415569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.415577 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.417702 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.418204 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.418208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.418208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.418333 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.418575 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.418576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.418577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.418857 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.418993 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.419276 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.419411 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":155,"thread_start_us":126,"threads_started":1}
I20260620 11:40:24.420125 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.408723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46734 (local address 127.5.122.190:37871)
0620 11:40:24.408797 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.408800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.409073 (+ 273us) server_negotiation.cc:407] Connection header received
0620 11:40:24.409729 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.409730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.409743 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.409793 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.410663 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.411489 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.412803 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.413014 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.413075 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.416848 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.416850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.416851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.416859 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.418698 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.419161 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.419163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.419163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.419221 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.419458 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.419460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.419460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.419609 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.419655 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.419909 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.420016 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 11:40:24.420274 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.408940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57258 (local address 127.5.122.188:45707)
0620 11:40:24.409120 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.409124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.409181 (+ 57us) server_negotiation.cc:407] Connection header received
0620 11:40:24.409637 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.409653 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.409685 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.409797 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.410632 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.411686 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.412479 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.412946 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.413064 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.416948 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.416950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.416950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.416965 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.418847 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.419328 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.419329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.419330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.419380 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.419612 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.419613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.419613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.419737 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.419850 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.420073 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.420157 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
I20260620 11:40:24.421017 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.408723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:37871 (local address 127.0.0.1:46734)
0620 11:40:24.408911 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.408954 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.409612 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.409876 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.409879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.410302 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.410581 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.410590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.411593 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.411595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.412724 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.412727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.413148 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.413149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.413228 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.414027 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.414039 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.416749 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.418803 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.418811 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.418813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.419079 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.419307 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.419308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.419308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.419378 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.419703 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.419711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.419758 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.420713 (+ 955us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.420879 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260620 11:40:24.422189 18965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: Bootstrap starting.
I20260620 11:40:24.422633 19115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:40:24.422983 18965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:24.423291 18965 log.cc:824] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:24.424144 18965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: No bootstrap required, opened a new log
I20260620 11:40:24.426258 18965 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.426402 18965 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:24.426465 18965 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab5c1eee06524baabb91c9785f74600a, State: Initialized, Role: FOLLOWER
I20260620 11:40:24.426611 18965 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.427008 18965 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:24.427259 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.427376 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [sys.catalog]: This master's current role is: FOLLOWER
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
W20260620 11:40:24.429157 19133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:40:24.429248 19133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:40:24.432106 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.423106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.5.122.189:44495)
0620 11:40:24.423513 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.423515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.423524 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:40:24.423545 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.423546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.423554 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.423635 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.423948 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.424636 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.426099 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.426459 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.426548 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.428795 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.428797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.428798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.428807 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.430776 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.431211 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.431213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.431213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.431275 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.431506 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.431507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.431507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.431653 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.431697 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.431913 (+ 216us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.432009 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380}
I20260620 11:40:24.432442 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.422083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.5.122.190:37871)
0620 11:40:24.422139 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.422142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.422152 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:40:24.422176 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.422176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.422186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.422244 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.423003 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.423627 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.424722 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.425002 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.425082 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.428261 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.428263 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.428263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.428270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.430318 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.430733 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.430734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.430735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.430796 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.431101 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.431102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.431103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.431249 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.431830 (+ 581us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.432255 (+ 425us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.432338 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:40:24.430804 18897 master_runner.cc:186] Error getting master registration for 127.5.122.189:44495: OK, instance_id { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" instance_seqno: 1781955624221639 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 11:40:24.433491 18897 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 11:40:24.433552 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.422941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:44495 (local address 127.0.0.1:40268)
0620 11:40:24.423076 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.423087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.423123 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.423639 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.423641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.423789 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.423852 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.423855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.425145 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.425146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.425934 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.425938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.426616 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.426618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.426706 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.427191 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.427197 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.428682 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.430892 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.430893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.430893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.431110 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.431349 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.431350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.431350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.431424 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.431748 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.431749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.431799 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.432741 (+ 942us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.433443 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260620 11:40:24.434095 19115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:40:24.441632 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.433197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.5.122.189:44495)
0620 11:40:24.433233 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.433235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.433261 (+ 26us) server_negotiation.cc:407] Connection header received
0620 11:40:24.433281 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.433282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.433294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.433344 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.434003 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.434722 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.435400 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.435603 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.435679 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.438278 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.438280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.438280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.438288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.439996 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.440372 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.440374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.440374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.440432 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.440653 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.440654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.440655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.440777 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.441407 (+ 630us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.441442 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.441505 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:40:24.442301 19117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.434347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57270)
0620 11:40:24.434455 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.434464 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.434509 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.434637 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.434638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.434786 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.434826 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.434828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.435730 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.435731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.436485 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.436487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.436995 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.436996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.437046 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.437474 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.437481 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.438989 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.440896 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.440897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.440898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.441101 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.441412 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.441413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.441413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.441477 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.441864 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.441865 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.441907 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.442056 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.442167 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260620 11:40:24.442345 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.434405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57270 (local address 127.5.122.188:45707)
0620 11:40:24.434441 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.434442 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.434512 (+ 70us) server_negotiation.cc:407] Connection header received
0620 11:40:24.434565 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.434566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.434577 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.434631 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.434909 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.435640 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.436579 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.436882 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.436922 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.439088 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.439089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.439090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.439096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.440771 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.441261 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.441262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.441262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.441322 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.441562 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.441564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.441564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.441766 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.441840 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.442103 (+ 263us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.442221 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":18}
I20260620 11:40:24.443496 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:40:24.444159 19115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288: Bootstrap starting.
I20260620 11:40:24.445014 19115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:24.445379 19115 log.cc:824] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:24.447494 19115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288: No bootstrap required, opened a new log
I20260620 11:40:24.449656 19115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.449851 19115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:24.449893 19115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df21a28d5154d89bd357903fc1b9288, State: Initialized, Role: FOLLOWER
I20260620 11:40:24.450095 19115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.450479 19115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:24.454480 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.445360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.5.122.188:45707)
0620 11:40:24.445412 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.445414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.445740 (+ 326us) server_negotiation.cc:407] Connection header received
0620 11:40:24.445799 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.445800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.445808 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.445856 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.446620 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.447367 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.448506 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.448806 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.448850 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.450978 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.450981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.450981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.450989 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.452878 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.453314 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.453315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.453315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.453377 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.453764 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.453765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.453766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.453908 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.454029 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.454264 (+ 235us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.454359 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 11:40:24.454753 19047 master_runner.cc:186] Error getting master registration for 127.5.122.189:44495: OK, instance_id { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" instance_seqno: 1781955624221639 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 11:40:24.457770 19047 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 11:40:24.458078 19143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.458209 19143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:40:24.458279 19143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:24.458348 19143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.458839 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9df21a28d5154d89bd357903fc1b9288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab5c1eee06524baabb91c9785f74600a" is_pre_election: true
I20260620 11:40:24.459059 18939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9df21a28d5154d89bd357903fc1b9288 in term 0.
I20260620 11:40:24.458946 19143 leader_election.cc:288] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab5c1eee06524baabb91c9785f74600a (127.5.122.190:37871), bbd0704fc5f141f78054feb9bad2a8bb (127.5.122.189:44495)
I20260620 11:40:24.459270 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.445545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.5.122.190:37871)
0620 11:40:24.445590 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.445592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.446207 (+ 615us) server_negotiation.cc:407] Connection header received
0620 11:40:24.446302 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.446303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.446314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.446346 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.451919 (+ 5573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.452680 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.453733 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.454057 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.454099 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.455619 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.455620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.455621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.455630 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.457373 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.457818 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.457819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.457820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.458088 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.458327 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.458328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.458329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.458753 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.458856 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.459085 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.459174 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:40:24.459296 19065 leader_election.cc:302] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9df21a28d5154d89bd357903fc1b9288, ab5c1eee06524baabb91c9785f74600a; no voters:
I20260620 11:40:24.459425 19040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } attempt: 3
I20260620 11:40:24.460114 19143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:40:24.460248 19143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:40:24.460316 19143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:24.461210 19143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.461349 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.445065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40278 (local address 127.5.122.189:44495)
0620 11:40:24.445663 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.445665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.445673 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:40:24.445695 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.445695 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.445705 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.445759 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.446619 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.447367 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.448500 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.448805 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.448850 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.451566 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.451567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.451568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.451576 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.459892 (+ 8316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.460284 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.460285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.460286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.460371 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.460596 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.460597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.460597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.460746 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.460847 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.461044 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.461175 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575}
I20260620 11:40:24.461557 19143 leader_election.cc:288] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [CANDIDATE]: Term 1 election: Requested vote from peers ab5c1eee06524baabb91c9785f74600a (127.5.122.190:37871), bbd0704fc5f141f78054feb9bad2a8bb (127.5.122.189:44495)
I20260620 11:40:24.461656 18939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9df21a28d5154d89bd357903fc1b9288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab5c1eee06524baabb91c9785f74600a"
W20260620 11:40:24.458942 19154 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:40:24.461768 19154 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:40:24.461777 18939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:24.463084 18939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9df21a28d5154d89bd357903fc1b9288 in term 1.
I20260620 11:40:24.463274 19065 leader_election.cc:302] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9df21a28d5154d89bd357903fc1b9288, ab5c1eee06524baabb91c9785f74600a; no voters:
I20260620 11:40:24.463363 19143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:24.463407 19143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [term 1 LEADER]: Becoming Leader. State: Replica: 9df21a28d5154d89bd357903fc1b9288, State: Running, Role: LEADER
I20260620 11:40:24.463518 19143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.464059 19143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df21a28d5154d89bd357903fc1b9288. Latest consensus state: current_term: 1 leader_uuid: "9df21a28d5154d89bd357903fc1b9288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.464147 19143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: This master's current role is: LEADER
Jun 20 11:40:24 dist-test-slave-szdr krb5kdc[18884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955624, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:40:24.467612 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.459649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:45707 (local address 127.0.0.1:57286)
0620 11:40:24.459719 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:24.459732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:24.459782 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:24.460054 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:24.460055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:24.460241 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:24.460295 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.460297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.461188 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.461189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.462008 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:24.462011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.462501 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.462502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:24.462558 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.463048 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:24.463053 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:24.464562 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:24.466523 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.466524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.466525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.466738 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.466928 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:24.466928 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:24.466929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:24.466983 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:24.467257 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:24.467258 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:24.467317 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 11:40:24.467404 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 11:40:24.467517 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 11:40:24.467612 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:24.459818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57286 (local address 127.5.122.188:45707)
0620 11:40:24.459880 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:24.459882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:24.459890 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:40:24.459922 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:24.459922 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:24.459932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:24.459986 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:24.460389 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.461088 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.462104 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:24.462377 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:24.462437 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:24.464643 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:24.464644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:24.464645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:24.464653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:24.466423 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.466818 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.466819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.466819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.466870 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:24.467064 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:24.467065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:24.467065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:24.467204 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:24.467266 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:24.467445 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 11:40:24.467517 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 11:40:24.469484 19040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: Bootstrap starting.
I20260620 11:40:24.470348 19040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:24.470687 19040 log.cc:824] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:24.471583 19040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: No bootstrap required, opened a new log
I20260620 11:40:24.472206 19158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:40:24.472424 19158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:24.473870 19040 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.474052 19040 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:24.474097 19040 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd0704fc5f141f78054feb9bad2a8bb, State: Initialized, Role: FOLLOWER
I20260620 11:40:24.474236 19040 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } }
I20260620 11:40:24.474578 19040 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:24.474606 19014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9df21a28d5154d89bd357903fc1b9288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" is_pre_election: true
I20260620 11:40:24.474740 19014 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9df21a28d5154d89bd357903fc1b9288 in term 0.
I20260620 11:40:24.474853 19013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9df21a28d5154d89bd357903fc1b9288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbd0704fc5f141f78054feb9bad2a8bb"
I20260620 11:40:24.474916 19013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:24.475886 19013 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9df21a28d5154d89bd357903fc1b9288 in term 1.
I20260620 11:40:24.474547 19161 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.476289 19161 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:40:24.482890 19013 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [term 1 FOLLOWER]: Refusing update from remote peer 9df21a28d5154d89bd357903fc1b9288: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:24.483042 18939 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [term 1 FOLLOWER]: Refusing update from remote peer 9df21a28d5154d89bd357903fc1b9288: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260620 11:40:24.483129 19172 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:40:24.483176 19172 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:40:24.483414 19143 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:24.483574 19155 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:24.484992 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df21a28d5154d89bd357903fc1b9288. Latest consensus state: current_term: 1 leader_uuid: "9df21a28d5154d89bd357903fc1b9288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.485100 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:40:24.485215 19161 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df21a28d5154d89bd357903fc1b9288. Latest consensus state: current_term: 1 leader_uuid: "9df21a28d5154d89bd357903fc1b9288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.485322 19161 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:40:24.487766 19143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9df21a28d5154d89bd357903fc1b9288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.487870 19143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:24.488337 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9df21a28d5154d89bd357903fc1b9288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.488417 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:40:24.488819 19155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9df21a28d5154d89bd357903fc1b9288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.488925 19155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288 [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:24.489053 19174 mvcc.cc:205] Tried to move back new op lower bound from 7298890237879644160 to 7298890237803433984. Current Snapshot: MvccSnapshot[applied={T|T < 7298890237879644160}]
I20260620 11:40:24.489152 19161 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9df21a28d5154d89bd357903fc1b9288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab5c1eee06524baabb91c9785f74600a" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37871 } } peers { permanent_uuid: "bbd0704fc5f141f78054feb9bad2a8bb" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 44495 } } peers { permanent_uuid: "9df21a28d5154d89bd357903fc1b9288" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 45707 } } }
I20260620 11:40:24.489233 19161 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:40:24.489322 19175 mvcc.cc:205] Tried to move back new op lower bound from 7298890237879644160 to 7298890237803433984. Current Snapshot: MvccSnapshot[applied={T|T < 7298890237879644160}]
I20260620 11:40:24.489444 19160 mvcc.cc:205] Tried to move back new op lower bound from 7298890237879644160 to 7298890237803433984. Current Snapshot: MvccSnapshot[applied={T|T < 7298890237879644160}]
I20260620 11:40:24.495106 19158 catalog_manager.cc:1347] Generated new cluster ID: 5d57989869bf4db787c83ebd93ff22f3
I20260620 11:40:24.495184 19158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:40:24.503494 19158 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:40:24.504323 19158 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:40:24.514189 19158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9df21a28d5154d89bd357903fc1b9288: Generated new TSK 0
I20260620 11:40:24.514472 19158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:40:25.429741 19133 catalog_manager.cc:1259] Loaded cluster ID: 5d57989869bf4db787c83ebd93ff22f3
I20260620 11:40:25.429826 19133 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: loading cluster ID for follower catalog manager: success
I20260620 11:40:25.430714 19133 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: acquiring CA information for follower catalog manager: success
I20260620 11:40:25.431027 19133 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ab5c1eee06524baabb91c9785f74600a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 11:40:25.483693 19172 catalog_manager.cc:1259] Loaded cluster ID: 5d57989869bf4db787c83ebd93ff22f3
I20260620 11:40:25.483786 19172 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: loading cluster ID for follower catalog manager: success
I20260620 11:40:25.484673 19172 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: acquiring CA information for follower catalog manager: success
I20260620 11:40:25.485013 19172 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbd0704fc5f141f78054feb9bad2a8bb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 11:40:25.569355 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18897
I20260620 11:40:25.574672 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 18972
I20260620 11:40:25.579798 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19047
2026-06-20T11:40:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2860 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:40:25 dist-test-slave-szdr krb5kdc[19189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:40:25 dist-test-slave-szdr krb5kdc[19189](info): set up 2 sockets
Jun 20 11:40:25 dist-test-slave-szdr krb5kdc[19189](info): commencing operation
krb5kdc: starting...
W20260620 11:40:26.642328 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:40:26 dist-test-slave-szdr krb5kdc[19189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:40:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:40:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:40:26.774682 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:40767
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:45843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:40767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:26.861052 19202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:26.861290 19202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:26.861315 19202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:26.862536 19202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:26.862577 19202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:26.862592 19202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:26.862605 19202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:26.862620 19202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:26.864135 19202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:45843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:40767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:26.864354 19202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:26.864545 19202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:26.870172 19207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:26.870175 19208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:26.870481 19202 server_base.cc:1034] running on GCE node
W20260620 11:40:26.870200 19210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:26.870868 19202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:26.871212 19202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:26.872406 19202 hybrid_clock.cc:648] HybridClock initialized: now 1781955626872399 us; error 60 us; skew 500 ppm
Jun 20 11:40:26 dist-test-slave-szdr krb5kdc[19189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955626, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:26.875278 19202 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:26.875778 19202 webserver.cc:460] Webserver started at http://127.5.122.190:39859/ using document root <none> and password file <none>
I20260620 11:40:26.876003 19202 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:26.876048 19202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:26.876143 19202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:26.877211 19202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d"
format_stamp: "Formatted at 2026-06-20 11:40:26 on dist-test-slave-szdr"
server_key: "0b5ac0fd8aa92c6b8ad4bd800eb8c592"
server_key_iv: "ae68a1251eb90f9785815fd26b690bb4"
server_key_version: "encryptionkey@0"
I20260620 11:40:26.877527 19202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d"
format_stamp: "Formatted at 2026-06-20 11:40:26 on dist-test-slave-szdr"
server_key: "0b5ac0fd8aa92c6b8ad4bd800eb8c592"
server_key_iv: "ae68a1251eb90f9785815fd26b690bb4"
server_key_version: "encryptionkey@0"
I20260620 11:40:26.879082 19202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:40:26.880000 19217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:26.880208 19202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:26.880327 19202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d"
format_stamp: "Formatted at 2026-06-20 11:40:26 on dist-test-slave-szdr"
server_key: "0b5ac0fd8aa92c6b8ad4bd800eb8c592"
server_key_iv: "ae68a1251eb90f9785815fd26b690bb4"
server_key_version: "encryptionkey@0"
I20260620 11:40:26.880410 19202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:26.894239 19202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:26.896894 19202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:26.897099 19202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:26.902253 19202 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:40767
I20260620 11:40:26.902307 19269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:40767 every 8 connection(s)
I20260620 11:40:26.902745 19202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:26.903822 19270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:26.906608 19270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d: Bootstrap starting.
I20260620 11:40:26.907310 19270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:26.907640 19270 log.cc:824] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:26.908599 19270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d: No bootstrap required, opened a new log
I20260620 11:40:26.910516 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19202
I20260620 11:40:26.910666 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:40:26.910923 5610 external_mini_cluster.cc:1428] Setting key 2170ead7a0830641a0fe97aa2492efb8
I20260620 11:40:26.910940 19270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40767 } }
I20260620 11:40:26.911167 19270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:26.911221 19270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee2cd2cfc82448099c0d8311bdd5c4d, State: Initialized, Role: FOLLOWER
I20260620 11:40:26.911396 19270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40767 } }
I20260620 11:40:26.911501 19270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:40:26.911542 19270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:40:26.911597 19270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:26.912328 19270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40767 } }
I20260620 11:40:26.912474 19270 leader_election.cc:302] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ee2cd2cfc82448099c0d8311bdd5c4d; no voters:
I20260620 11:40:26.912668 19270 leader_election.cc:288] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:40:26.912707 19275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:26.912868 19275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [term 1 LEADER]: Becoming Leader. State: Replica: 6ee2cd2cfc82448099c0d8311bdd5c4d, State: Running, Role: LEADER
I20260620 11:40:26.912979 19275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40767 } }
I20260620 11:40:26.912984 19270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:26.914866 19276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40767 } } }
I20260620 11:40:26.914844 19277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee2cd2cfc82448099c0d8311bdd5c4d. Latest consensus state: current_term: 1 leader_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee2cd2cfc82448099c0d8311bdd5c4d" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40767 } } }
I20260620 11:40:26.915000 19276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:26.915009 19277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:26.915232 19290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:40:26.916095 19290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:26.917819 19290 catalog_manager.cc:1347] Generated new cluster ID: 6fe0b68c14974361a54c90f911f177b2
I20260620 11:40:26.917881 19290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:40:26 dist-test-slave-szdr krb5kdc[19189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:26.925475 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:26.911528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.5.122.190:40767)
0620 11:40:26.911774 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:26.911784 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:26.911821 (+ 37us) server_negotiation.cc:407] Connection header received
0620 11:40:26.911941 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:26.911947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:26.912183 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:26.912351 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:26.913273 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:26.915222 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:26.916349 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:26.917034 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:26.917156 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:26.920586 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:26.920591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:26.920592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:26.920607 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:26.923047 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:26.923531 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:26.923535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:26.923535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:26.923676 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:26.923900 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:26.923901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:26.923901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:26.924318 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:26.924517 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:26.925026 (+ 509us) server_negotiation.cc:299] Negotiation successful
0620 11:40:26.925192 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":102,"threads_started":1}
I20260620 11:40:26.926817 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:40:26.934316 19290 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:40:26.934955 19290 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:40:26.948284 19290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ee2cd2cfc82448099c0d8311bdd5c4d: Generated new TSK 0
I20260620 11:40:26.948581 19290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:40:26.981472 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19202
2026-06-20T11:40:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1398 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:40:27 dist-test-slave-szdr krb5kdc[19297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:40:27 dist-test-slave-szdr krb5kdc[19297](info): set up 2 sockets
Jun 20 11:40:27 dist-test-slave-szdr krb5kdc[19297](info): commencing operation
krb5kdc: starting...
W20260620 11:40:28.042356 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:40:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:40:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:40:28.169381 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39689
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:28.254408 19310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:28.254655 19310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:28.254681 19310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:28.255927 19310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:28.255978 19310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:28.255993 19310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:28.256006 19310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:28.256019 19310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:28.257560 19310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:28.257777 19310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:28.257994 19310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:28.263265 19316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:28.263289 19318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:28.263288 19315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:28.263700 19310 server_base.cc:1034] running on GCE node
I20260620 11:40:28.263970 19310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:28.264288 19310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:28.265465 19310 hybrid_clock.cc:648] HybridClock initialized: now 1781955628265447 us; error 28 us; skew 500 ppm
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:28.274473 19310 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:28.274974 19310 webserver.cc:460] Webserver started at http://127.5.122.190:43347/ using document root <none> and password file <none>
I20260620 11:40:28.275233 19310 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:28.275316 19310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:28.275466 19310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:28.276547 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "22536e15c20e4537bb7ddfc8967e1a8f"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "41b76673e0b3073a659f53dce9fabfc8"
server_key_iv: "1ecb7a0e4dd79035245af55dee01211e"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.276909 19310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "22536e15c20e4537bb7ddfc8967e1a8f"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "41b76673e0b3073a659f53dce9fabfc8"
server_key_iv: "1ecb7a0e4dd79035245af55dee01211e"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.278295 19310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:40:28.279079 19325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.279358 19310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:28.279467 19310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "22536e15c20e4537bb7ddfc8967e1a8f"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "41b76673e0b3073a659f53dce9fabfc8"
server_key_iv: "1ecb7a0e4dd79035245af55dee01211e"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.279546 19310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:28.289444 19310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:28.291543 19310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:28.291735 19310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:28.296062 19310 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:39689
I20260620 11:40:28.296096 19377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:39689 every 8 connection(s)
I20260620 11:40:28.296501 19310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:28.297569 19378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:28.300103 19378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Bootstrap starting.
I20260620 11:40:28.300769 19378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:28.301080 19378 log.cc:824] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:28.301875 19378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: No bootstrap required, opened a new log
I20260620 11:40:28.303772 19378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:28.303954 19378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:28.303982 19378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22536e15c20e4537bb7ddfc8967e1a8f, State: Initialized, Role: FOLLOWER
I20260620 11:40:28.304191 19378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:28.304284 19378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:40:28.304311 19378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:40:28.304363 19378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:28.304940 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19310
I20260620 11:40:28.305081 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:40:28.305351 5610 external_mini_cluster.cc:1428] Setting key 6b9d4c59ca992d104fb579f6c3d095e2
I20260620 11:40:28.305639 19378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:28.305773 19378 leader_election.cc:302] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22536e15c20e4537bb7ddfc8967e1a8f; no voters:
I20260620 11:40:28.306015 19378 leader_election.cc:288] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:40:28.306037 19382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:28.306149 19382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 LEADER]: Becoming Leader. State: Replica: 22536e15c20e4537bb7ddfc8967e1a8f, State: Running, Role: LEADER
I20260620 11:40:28.306226 19382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:28.306308 19378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:28.306587 19382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } } }
I20260620 11:40:28.306663 19382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:28.306788 19382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22536e15c20e4537bb7ddfc8967e1a8f. Latest consensus state: current_term: 1 leader_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } } }
I20260620 11:40:28.306861 19382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:28.307159 19388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:40:28.307742 19388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:28.309682 19388 catalog_manager.cc:1347] Generated new cluster ID: a6684628ada543b7977a81979eeff5ac
I20260620 11:40:28.309744 19388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:28.319281 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.306184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.5.122.190:39689)
0620 11:40:28.306461 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.306470 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.306499 (+ 29us) server_negotiation.cc:407] Connection header received
0620 11:40:28.306696 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.306733 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.306804 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.307935 (+ 1131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:28.308093 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.309315 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.310481 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.311133 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.311254 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.314390 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:28.314396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:28.314397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:28.314413 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:28.317254 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.317765 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.317770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.317770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.317931 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.318189 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.318190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.318191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.318576 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:28.318748 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.318863 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.319025 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":78,"threads_started":1}
I20260620 11:40:28.325907 19388 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:40:28.326930 19388 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:40:28.330629 19388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Generated new TSK 0
I20260620 11:40:28.330909 19388 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:40:28.361555 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39689
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:28.447705 19404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:28.447948 19404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:28.447978 19404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:28.449316 19404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:28.449376 19404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:28.449409 19404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:40:28.451121 19404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:28.451434 19404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:28.451709 19404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:40:28.457597 19404 server_base.cc:1034] running on GCE node
W20260620 11:40:28.457491 19410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:28.457491 19409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:28.457492 19412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:28.458077 19404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:28.458402 19404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:28.459591 19404 hybrid_clock.cc:648] HybridClock initialized: now 1781955628459577 us; error 32 us; skew 500 ppm
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:28.462566 19404 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:40:28.463099 19404 webserver.cc:460] Webserver started at http://127.5.122.129:35293/ using document root <none> and password file <none>
I20260620 11:40:28.463318 19404 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:28.463363 19404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:28.463472 19404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:28.464546 19404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "56e88bc305f44e5287a07699fc31edbe"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "439f8ef7456cd4b7cc51d5388d28c5ff"
server_key_iv: "13d914580e2d6c3a7a73c114cb1d9cb9"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.464864 19404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "56e88bc305f44e5287a07699fc31edbe"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "439f8ef7456cd4b7cc51d5388d28c5ff"
server_key_iv: "13d914580e2d6c3a7a73c114cb1d9cb9"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.466326 19404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:40:28.467222 19419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.467425 19404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:28.467521 19404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "56e88bc305f44e5287a07699fc31edbe"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "439f8ef7456cd4b7cc51d5388d28c5ff"
server_key_iv: "13d914580e2d6c3a7a73c114cb1d9cb9"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.467588 19404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:28.489002 19404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:28.491597 19404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:28.491726 19404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:28.491976 19404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:28.492343 19404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:28.492379 19404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.492408 19404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:28.492458 19404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.498663 19404 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:40161
I20260620 11:40:28.498745 19532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:40161 every 8 connection(s)
I20260620 11:40:28.499138 19404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:28.507354 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19404
I20260620 11:40:28.507519 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:40:28.507769 5610 external_mini_cluster.cc:1428] Setting key 69b5a4dd6f46fe9de67bff12a702efd5
I20260620 11:40:28.510372 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.500008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:56557 (local address 127.5.122.190:39689)
0620 11:40:28.500081 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.500083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.500262 (+ 179us) server_negotiation.cc:407] Connection header received
0620 11:40:28.500830 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.500832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.500849 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.500924 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:28.502080 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.502616 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.503732 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.503976 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.504060 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.506961 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:28.506963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:28.506963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:28.506980 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:28.508694 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.509192 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.509194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.509194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.509265 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.509558 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.509559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.509560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.509691 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:28.509791 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.510165 (+ 374us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.510265 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 11:40:28.510620 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.499921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:56557)
0620 11:40:28.500104 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.500162 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.500712 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.501051 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.501055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.501687 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:28.501909 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.501915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.502723 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.502724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.503625 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.503628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.504071 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.504072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.504215 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.504807 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:28.504817 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:28.506781 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:28.508818 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:28.508829 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:28.508833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:28.509109 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:28.509341 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:28.509342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:28.509342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:28.509479 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:28.509784 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:28.509793 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:28.510003 (+ 210us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.510140 (+ 137us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.510346 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260620 11:40:28.511157 19533 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39689
I20260620 11:40:28.511277 19533 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:28.511503 19533 heartbeater.cc:511] Master 127.5.122.190:39689 requested a full tablet report, sending...
I20260620 11:40:28.512072 19342 ts_manager.cc:194] Registered new tserver with Master: 56e88bc305f44e5287a07699fc31edbe (127.5.122.129:40161)
I20260620 11:40:28.512746 19342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:56557
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:40:28.536573 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39689
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:28.625058 19538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:28.625308 19538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:28.625339 19538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:28.626694 19538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:28.626745 19538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:28.626775 19538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:40:28.628415 19538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:28.628649 19538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:28.628852 19538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:28.634356 19543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:28.634512 19544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:28.634440 19538 server_base.cc:1034] running on GCE node
W20260620 11:40:28.634392 19546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:28.634877 19538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:28.635185 19538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:28.636363 19538 hybrid_clock.cc:648] HybridClock initialized: now 1781955628636329 us; error 54 us; skew 500 ppm
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:28.639124 19538 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:40:28.639657 19538 webserver.cc:460] Webserver started at http://127.5.122.130:33487/ using document root <none> and password file <none>
I20260620 11:40:28.639868 19538 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:28.639909 19538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:28.640012 19538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:28.641057 19538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "61f366cafe5c4f4fadc5e69121d4a6b8"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "27a704337b643ac65aa3040ebe54f505"
server_key_iv: "207a125ffaa1df701013b64234b42f0c"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.641371 19538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "61f366cafe5c4f4fadc5e69121d4a6b8"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "27a704337b643ac65aa3040ebe54f505"
server_key_iv: "207a125ffaa1df701013b64234b42f0c"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.642818 19538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:40:28.643647 19553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.643900 19538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:28.644006 19538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "61f366cafe5c4f4fadc5e69121d4a6b8"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "27a704337b643ac65aa3040ebe54f505"
server_key_iv: "207a125ffaa1df701013b64234b42f0c"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.644070 19538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:28.656435 19538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:28.658994 19538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:28.659118 19538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:28.659319 19538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:28.659665 19538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:28.659703 19538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.659729 19538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:28.659741 19538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.665851 19538 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:41515
I20260620 11:40:28.665956 19666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:41515 every 8 connection(s)
I20260620 11:40:28.666323 19538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:40:28.672271 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19538
I20260620 11:40:28.672430 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:40:28.672662 5610 external_mini_cluster.cc:1428] Setting key 0d8d2e19514e10ec70892e24947edf2f
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:28.677032 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.667254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:44435 (local address 127.5.122.190:39689)
0620 11:40:28.667349 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.667352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.667521 (+ 169us) server_negotiation.cc:407] Connection header received
0620 11:40:28.668046 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.668048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.668059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.668125 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:28.669168 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.669670 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.671023 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.671222 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.671288 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.673686 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:28.673687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:28.673687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:28.673695 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:28.675340 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.675934 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.675935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.675935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.675990 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.676295 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.676295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.676296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.676408 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:28.676461 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.676852 (+ 391us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.676963 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 11:40:28.677353 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.667192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:44435)
0620 11:40:28.667371 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.667420 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.667933 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.668197 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.668201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.668797 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:28.669065 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.669072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.669787 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.669788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.670929 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.670933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.671343 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.671344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.671465 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.672038 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:28.672047 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:28.673591 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:28.675435 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:28.675443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:28.675446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:28.675807 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:28.676061 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:28.676062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:28.676062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:28.676228 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:28.676475 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:28.676481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:28.676690 (+ 209us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.676835 (+ 145us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.677036 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260620 11:40:28.677922 19667 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39689
I20260620 11:40:28.678071 19667 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:28.678265 19667 heartbeater.cc:511] Master 127.5.122.190:39689 requested a full tablet report, sending...
I20260620 11:40:28.678740 19342 ts_manager.cc:194] Registered new tserver with Master: 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515)
I20260620 11:40:28.679134 19342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:44435
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:40:28.701385 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39689
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:28.790088 19672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:28.790316 19672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:28.790343 19672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:28.791635 19672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:28.791689 19672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:28.791718 19672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:40:28.793319 19672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:28.793620 19672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:28.793876 19672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:28.799990 19680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:28.800015 19677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:28.800141 19672 server_base.cc:1034] running on GCE node
W20260620 11:40:28.800211 19678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:28.800505 19672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:28.800822 19672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:28.802030 19672 hybrid_clock.cc:648] HybridClock initialized: now 1781955628802003 us; error 33 us; skew 500 ppm
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:28.804723 19672 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:40:28.805238 19672 webserver.cc:460] Webserver started at http://127.5.122.131:33255/ using document root <none> and password file <none>
I20260620 11:40:28.805498 19672 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:28.805573 19672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:28.805725 19672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:28.807140 19672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "cc377929f0004133b7638929ef1d288a"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "5f3b8fb14734652d7b6de5bc34ca303b"
server_key_iv: "081e5e2abfe2c51478258dbc62dee1e6"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.807585 19672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "cc377929f0004133b7638929ef1d288a"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "5f3b8fb14734652d7b6de5bc34ca303b"
server_key_iv: "081e5e2abfe2c51478258dbc62dee1e6"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.809147 19672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:40:28.810196 19687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.810415 19672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:28.810523 19672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "cc377929f0004133b7638929ef1d288a"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "5f3b8fb14734652d7b6de5bc34ca303b"
server_key_iv: "081e5e2abfe2c51478258dbc62dee1e6"
server_key_version: "encryptionkey@0"
I20260620 11:40:28.810613 19672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:28.820911 19672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:28.823653 19672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:28.823848 19672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:28.824126 19672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:28.824481 19672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:28.824517 19672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.824575 19672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:28.824604 19672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:28.831372 19672 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:36641
I20260620 11:40:28.831459 19800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:36641 every 8 connection(s)
I20260620 11:40:28.831836 19672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:40:28.837263 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19672
I20260620 11:40:28.837381 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:40:28.837603 5610 external_mini_cluster.cc:1428] Setting key 7511a59b6d1e4f075147cf961ee01a11
Jun 20 11:40:28 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955628, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:28.842262 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.832646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:53885 (local address 127.5.122.190:39689)
0620 11:40:28.832711 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.832713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.832914 (+ 201us) server_negotiation.cc:407] Connection header received
0620 11:40:28.833420 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.833421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.833431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.833495 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:28.834623 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.835344 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.836443 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.836631 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.836687 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.839439 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:28.839441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:28.839441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:28.839451 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:28.840854 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.841273 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.841274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.841274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.841328 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.841614 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.841615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.841615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.841723 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:28.841774 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.842117 (+ 343us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.842185 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:40:28.842526 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.832634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:53885)
0620 11:40:28.832783 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.832826 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.833328 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.833554 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.833557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.834260 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:28.834483 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.834489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.835464 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.835465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.836349 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.836353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.836713 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.836714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.836827 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.837489 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:28.837508 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:28.839350 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:28.840947 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:28.840955 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:28.840957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:28.841205 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:28.841424 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:28.841425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:28.841425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:28.841551 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:28.841784 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:28.841789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:28.841923 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.842095 (+ 172us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.842252 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
I20260620 11:40:28.843070 19801 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39689
I20260620 11:40:28.843195 19801 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:28.843407 19801 heartbeater.cc:511] Master 127.5.122.190:39689 requested a full tablet report, sending...
I20260620 11:40:28.843971 19342 ts_manager.cc:194] Registered new tserver with Master: cc377929f0004133b7638929ef1d288a (127.5.122.131:36641)
I20260620 11:40:28.844362 19342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:53885
I20260620 11:40:28.849745 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:40:28.857677 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.851208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.5.122.190:39689)
0620 11:40:28.851297 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.851299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.851378 (+ 79us) server_negotiation.cc:407] Connection header received
0620 11:40:28.851421 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.851421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.851431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.851479 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:28.852041 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.852550 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.853298 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.853495 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.853587 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.855015 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:28.855017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:28.855017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:28.855025 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:28.856409 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.856775 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.856777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.856777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.856842 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:28.857067 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:28.857068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:28.857068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:28.857235 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:28.857344 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.857489 (+ 145us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.857573 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 11:40:28.859587 19342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:40:28.861270 19342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:40:28.868896 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.864284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:36641 (local address 127.0.0.1:46592)
0620 11:40:28.864680 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.864691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.864733 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.865059 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.865060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.865062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:28.865115 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.865117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.866475 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.866475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.867650 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.867653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.868538 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.868539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.868633 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.868652 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.868733 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.868799 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":10,"thread_start_us":81,"threads_started":1}
I20260620 11:40:28.868924 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.864450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46592 (local address 127.5.122.131:36641)
0620 11:40:28.864864 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.864868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.864898 (+ 30us) server_negotiation.cc:407] Connection header received
0620 11:40:28.864937 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.864939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.864956 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.865041 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:28.865205 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.866366 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.867760 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.868409 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.868536 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.868622 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.868775 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.868832 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":125,"threads_started":1}
I20260620 11:40:28.869781 19813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.864039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:40161 (local address 127.0.0.1:39716)
0620 11:40:28.864433 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.864503 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.864566 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.864932 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.864935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.864937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:28.865025 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.865027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.867764 (+ 2737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.867765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.868476 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.868478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.869509 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.869510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.869595 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.869599 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.869651 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.869707 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":81,"threads_started":1}
I20260620 11:40:28.870178 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.864166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.5.122.129:40161)
0620 11:40:28.864742 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.864748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.864759 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:40:28.864803 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.864805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.864823 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.864908 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:28.865118 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.866366 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.868579 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.869409 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.869490 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.869555 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.870036 (+ 481us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.870091 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":106,"threads_started":1}
I20260620 11:40:28.870186 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.864038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41515 (local address 127.0.0.1:37102)
0620 11:40:28.864433 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.864482 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.864560 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.865250 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.865250 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.865252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:28.865291 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.865292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.866238 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.866239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.867650 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.867653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.869862 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.869863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.869931 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.869935 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.870025 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.870119 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":7,"spinlock_wait_cycles":896,"thread_start_us":110,"threads_started":1}
I20260620 11:40:28.870225 19735 tablet_service.cc:1467] Processing CreateTablet for tablet 0a961fcdb0384155909717cede49cf89 (DEFAULT_TABLE table=test-table [id=af4f6c6c760045d68faeef7c18361be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:28.870715 19735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a961fcdb0384155909717cede49cf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:28.870738 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.864265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.5.122.130:41515)
0620 11:40:28.865096 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.865101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.865111 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:40:28.865155 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.865157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.865175 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.865246 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:28.865341 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.866131 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.868666 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.869763 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.869861 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.869920 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.870592 (+ 672us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.870647 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":81,"threads_started":1}
I20260620 11:40:28.871008 19467 tablet_service.cc:1467] Processing CreateTablet for tablet 0a961fcdb0384155909717cede49cf89 (DEFAULT_TABLE table=test-table [id=af4f6c6c760045d68faeef7c18361be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:28.871402 19467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a961fcdb0384155909717cede49cf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:28.871436 19601 tablet_service.cc:1467] Processing CreateTablet for tablet 0a961fcdb0384155909717cede49cf89 (DEFAULT_TABLE table=test-table [id=af4f6c6c760045d68faeef7c18361be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:28.871754 19601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a961fcdb0384155909717cede49cf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:28.874365 19818 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap starting.
I20260620 11:40:28.875057 19818 tablet_bootstrap.cc:654] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:28.875173 19820 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Bootstrap starting.
I20260620 11:40:28.875489 19819 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Bootstrap starting.
I20260620 11:40:28.875876 19820 tablet_bootstrap.cc:654] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:28.875970 19818 log.cc:824] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:28.876246 19819 tablet_bootstrap.cc:654] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:28.876542 19820 log.cc:824] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:28.876832 19818 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: No bootstrap required, opened a new log
I20260620 11:40:28.876843 19819 log.cc:824] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:28.876921 19818 ts_tablet_manager.cc:1397] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:40:28.877337 19820 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: No bootstrap required, opened a new log
I20260620 11:40:28.877436 19820 ts_tablet_manager.cc:1397] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:40:28.878630 19819 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: No bootstrap required, opened a new log
I20260620 11:40:28.878712 19819 ts_tablet_manager.cc:1397] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:40:28.878841 19818 raft_consensus.cc:348] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.879038 19818 raft_consensus.cc:374] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:28.879074 19818 raft_consensus.cc:729] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e88bc305f44e5287a07699fc31edbe, State: Initialized, Role: FOLLOWER
I20260620 11:40:28.879222 19818 consensus_queue.cc:260] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.879215 19820 raft_consensus.cc:348] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.879437 19820 raft_consensus.cc:374] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:28.879489 19820 raft_consensus.cc:729] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f366cafe5c4f4fadc5e69121d4a6b8, State: Initialized, Role: FOLLOWER
I20260620 11:40:28.879617 19820 consensus_queue.cc:260] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.879935 19818 ts_tablet_manager.cc:1428] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 11:40:28.879961 19820 ts_tablet_manager.cc:1428] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 11:40:28.880100 19667 heartbeater.cc:503] Master 127.5.122.190:39689 was elected leader, sending a full tablet report...
I20260620 11:40:28.880138 19533 heartbeater.cc:503] Master 127.5.122.190:39689 was elected leader, sending a full tablet report...
I20260620 11:40:28.880846 19819 raft_consensus.cc:348] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.881031 19819 raft_consensus.cc:374] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:28.881079 19819 raft_consensus.cc:729] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc377929f0004133b7638929ef1d288a, State: Initialized, Role: FOLLOWER
I20260620 11:40:28.881197 19819 consensus_queue.cc:260] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.881485 19819 ts_tablet_manager.cc:1428] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:40:28.881577 19801 heartbeater.cc:503] Master 127.5.122.190:39689 was elected leader, sending a full tablet report...
W20260620 11:40:28.916988 19668 tablet.cc:2369] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:40:28.948629 19824 raft_consensus.cc:482] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:28.948782 19824 raft_consensus.cc:504] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.949188 19824 leader_election.cc:288] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515), cc377929f0004133b7638929ef1d288a (127.5.122.131:36641)
I20260620 11:40:28.952909 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.949566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:57779 (local address 127.5.122.130:41515)
0620 11:40:28.949618 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.949620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.949662 (+ 42us) server_negotiation.cc:407] Connection header received
0620 11:40:28.949766 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.949767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.949779 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.949918 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:28.950143 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.951006 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.952114 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.952513 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.952608 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.952619 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.952785 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.952826 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:40:28.952926 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.949507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41515 (local address 127.5.122.129:57779)
0620 11:40:28.949597 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.949612 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.949680 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.949945 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.949945 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.949948 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:28.950057 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.950060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.951125 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.951126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.952007 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.952010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.952605 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.952606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.952689 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.952692 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.952759 (+ 67us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.952807 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":22}
I20260620 11:40:28.952924 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.949788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:53211 (local address 127.5.122.131:36641)
0620 11:40:28.949840 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.949843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.950126 (+ 283us) server_negotiation.cc:407] Connection header received
0620 11:40:28.950158 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.950159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.950170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.950233 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:28.950369 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.951108 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.952124 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.952543 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.952636 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.952645 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.952803 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.952850 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:40:28.952945 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.949701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:36641 (local address 127.5.122.129:53211)
0620 11:40:28.950033 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:28.950044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:28.950104 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:28.950244 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:28.950244 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:28.950246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:28.950315 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.950316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.951177 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.951179 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.952023 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:28.952026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.952636 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.952637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:28.952715 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.952719 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:40:28.952798 (+ 79us) client_negotiation.cc:240] Negotiation successful
0620 11:40:28.952849 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":245,"threads_started":1}
I20260620 11:40:28.953367 19621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a961fcdb0384155909717cede49cf89" candidate_uuid: "56e88bc305f44e5287a07699fc31edbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" is_pre_election: true
I20260620 11:40:28.953583 19621 raft_consensus.cc:2454] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e88bc305f44e5287a07699fc31edbe in term 0.
I20260620 11:40:28.953531 19755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a961fcdb0384155909717cede49cf89" candidate_uuid: "56e88bc305f44e5287a07699fc31edbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc377929f0004133b7638929ef1d288a" is_pre_election: true
I20260620 11:40:28.953723 19755 raft_consensus.cc:2454] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56e88bc305f44e5287a07699fc31edbe in term 0.
I20260620 11:40:28.953888 19423 leader_election.cc:302] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe, 61f366cafe5c4f4fadc5e69121d4a6b8; no voters:
I20260620 11:40:28.954075 19824 raft_consensus.cc:2781] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:40:28.954167 19824 raft_consensus.cc:482] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:40:28.954201 19824 raft_consensus.cc:3037] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:28.955055 19824 raft_consensus.cc:504] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.955217 19824 leader_election.cc:288] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 1 election: Requested vote from peers 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515), cc377929f0004133b7638929ef1d288a (127.5.122.131:36641)
I20260620 11:40:28.955435 19621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a961fcdb0384155909717cede49cf89" candidate_uuid: "56e88bc305f44e5287a07699fc31edbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8"
I20260620 11:40:28.955495 19755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a961fcdb0384155909717cede49cf89" candidate_uuid: "56e88bc305f44e5287a07699fc31edbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc377929f0004133b7638929ef1d288a"
I20260620 11:40:28.955535 19621 raft_consensus.cc:3037] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:28.955595 19755 raft_consensus.cc:3037] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:28.956362 19755 raft_consensus.cc:2454] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e88bc305f44e5287a07699fc31edbe in term 1.
I20260620 11:40:28.956389 19621 raft_consensus.cc:2454] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56e88bc305f44e5287a07699fc31edbe in term 1.
I20260620 11:40:28.956594 19423 leader_election.cc:302] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe, 61f366cafe5c4f4fadc5e69121d4a6b8; no voters:
I20260620 11:40:28.956830 19824 raft_consensus.cc:2781] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:28.957026 19824 raft_consensus.cc:686] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 LEADER]: Becoming Leader. State: Replica: 56e88bc305f44e5287a07699fc31edbe, State: Running, Role: LEADER
I20260620 11:40:28.957139 19824 consensus_queue.cc:237] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:28.958132 19341 catalog_manager.cc:5495] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe reported cstate change: term changed from 0 to 1, leader changed from <none> to 56e88bc305f44e5287a07699fc31edbe (127.5.122.129). New cstate: current_term: 1 leader_uuid: "56e88bc305f44e5287a07699fc31edbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } health_report { overall_health: HEALTHY } } }
I20260620 11:40:28.985707 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:28.983094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.5.122.129:40161)
0620 11:40:28.983171 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:28.983173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:28.983182 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:40:28.983227 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:28.983228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:28.983247 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:28.983330 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:40:28.983524 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.984032 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.984838 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:28.985036 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:28.985114 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:28.985321 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:40:28.985443 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:40:28.985538 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:28.985566 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 11:40:28.985589 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 11:40:28.987783 19755 raft_consensus.cc:1261] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 1 FOLLOWER]: Refusing update from remote peer 56e88bc305f44e5287a07699fc31edbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:28.987803 19621 raft_consensus.cc:1261] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Refusing update from remote peer 56e88bc305f44e5287a07699fc31edbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:28.988252 19824 consensus_queue.cc:1035] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:28.988464 19828 consensus_queue.cc:1035] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:28.991668 19833 mvcc.cc:205] Tried to move back new op lower bound from 7298890256330907648 to 7298890256209186816. Current Snapshot: MvccSnapshot[applied={T|T < 7298890256330907648}]
I20260620 11:40:28.991725 19832 mvcc.cc:205] Tried to move back new op lower bound from 7298890256330907648 to 7298890256209186816. Current Snapshot: MvccSnapshot[applied={T|T < 7298890256330907648}]
I20260620 11:40:28.993624 19341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:28.993743 19341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:28.994562 19341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Sending DeleteTablet for 3 replicas of tablet 0a961fcdb0384155909717cede49cf89
I20260620 11:40:28.994918 19601 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a961fcdb0384155909717cede49cf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:28 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37102
I20260620 11:40:28.994937 19735 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a961fcdb0384155909717cede49cf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:28 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:46592
I20260620 11:40:28.994928 19467 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a961fcdb0384155909717cede49cf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:28 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:39716
I20260620 11:40:28.995162 19843 tablet_replica.cc:330] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: stopping tablet replica
I20260620 11:40:28.995232 19842 tablet_replica.cc:330] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: stopping tablet replica
I20260620 11:40:28.995270 19843 raft_consensus.cc:2229] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:40:28.995330 19842 raft_consensus.cc:2229] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:40:28.995381 19843 raft_consensus.cc:2258] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:28.995445 19842 raft_consensus.cc:2258] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:28.995446 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19404
I20260620 11:40:28.995591 19841 tablet_replica.cc:330] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: stopping tablet replica
I20260620 11:40:28.995682 19841 raft_consensus.cc:2229] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:40:28.995777 19841 raft_consensus.cc:2258] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:28.996366 19841 ts_tablet_manager.cc:1905] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:28.996666 19843 ts_tablet_manager.cc:1905] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:28.998483 19843 ts_tablet_manager.cc:1918] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:28.998564 19843 log.cc:1196] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/0a961fcdb0384155909717cede49cf89
I20260620 11:40:28.998561 19841 ts_tablet_manager.cc:1918] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:28.998641 19841 log.cc:1196] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/0a961fcdb0384155909717cede49cf89
I20260620 11:40:28.998876 19843 ts_tablet_manager.cc:1939] T 0a961fcdb0384155909717cede49cf89 P 61f366cafe5c4f4fadc5e69121d4a6b8: Deleting consensus metadata
I20260620 11:40:28.998940 19841 ts_tablet_manager.cc:1939] T 0a961fcdb0384155909717cede49cf89 P cc377929f0004133b7638929ef1d288a: Deleting consensus metadata
I20260620 11:40:28.999392 19329 catalog_manager.cc:4841] TS cc377929f0004133b7638929ef1d288a (127.5.122.131:36641): tablet 0a961fcdb0384155909717cede49cf89 (table test-table [id=af4f6c6c760045d68faeef7c18361be1]) successfully deleted
I20260620 11:40:28.999502 19329 catalog_manager.cc:4841] TS 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515): tablet 0a961fcdb0384155909717cede49cf89 (table test-table [id=af4f6c6c760045d68faeef7c18361be1]) successfully deleted
W20260620 11:40:29.002460 19327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:40161 (error 108)
I20260620 11:40:29.003134 19812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.002989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:40161 (local address 127.0.0.1:39746)
0620 11:40:29.003053 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.003094 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:40161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 11:40:29.003211 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19538
W20260620 11:40:29.003278 19327 catalog_manager.cc:4584] TS 56e88bc305f44e5287a07699fc31edbe (127.5.122.129:40161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a961fcdb0384155909717cede49cf89: Network error: Client connection negotiation failed: client connection to 127.5.122.129:40161: connect: Connection refused (error 111)
I20260620 11:40:29.008504 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19672
I20260620 11:40:29.013695 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19310
I20260620 11:40:29.019876 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39689
--webserver_interface=127.5.122.190
--webserver_port=43347
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:29.106323 19845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:29.106570 19845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:29.106603 19845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:29.107949 19845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:29.108016 19845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:29.108031 19845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:29.108049 19845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:29.108062 19845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:29.109732 19845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=43347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:29.110033 19845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:29.110289 19845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:29.119903 19854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.119932 19852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.119932 19851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:29.120136 19845 server_base.cc:1034] running on GCE node
I20260620 11:40:29.120539 19845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:29.120978 19845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:29.122131 19845 hybrid_clock.cc:648] HybridClock initialized: now 1781955629122119 us; error 35 us; skew 500 ppm
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:29.124984 19845 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:29.125572 19845 webserver.cc:460] Webserver started at http://127.5.122.190:43347/ using document root <none> and password file <none>
I20260620 11:40:29.125864 19845 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:29.125968 19845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:29.127532 19845 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:29.128350 19861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.128552 19845 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:29.128664 19845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "22536e15c20e4537bb7ddfc8967e1a8f"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "41b76673e0b3073a659f53dce9fabfc8"
server_key_iv: "1ecb7a0e4dd79035245af55dee01211e"
server_key_version: "encryptionkey@0"
I20260620 11:40:29.129202 19845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:29.140336 19845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:29.142486 19845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:29.142671 19845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:29.147378 19845 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:39689
I20260620 11:40:29.147431 19913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:39689 every 8 connection(s)
I20260620 11:40:29.147925 19845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:29.149722 19914 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:40:29.150606 19914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Bootstrap starting.
I20260620 11:40:29.156008 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19845
I20260620 11:40:29.156412 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:40161
--local_ip_for_outbound_sockets=127.5.122.129
--tserver_master_addrs=127.5.122.190:39689
--webserver_port=35293
--webserver_interface=127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:29.160142 19914 log.cc:824] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:29.162452 19914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:40:29.162828 19914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Bootstrap complete.
I20260620 11:40:29.165287 19914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:29.165567 19914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22536e15c20e4537bb7ddfc8967e1a8f, State: Initialized, Role: FOLLOWER
I20260620 11:40:29.165763 19914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:29.165854 19914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:40:29.165920 19914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:40:29.165999 19914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 1 FOLLOWER]: Advancing to term 2
I20260620 11:40:29.167218 19914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:29.167320 19914 leader_election.cc:302] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22536e15c20e4537bb7ddfc8967e1a8f; no voters:
I20260620 11:40:29.167559 19914 leader_election.cc:288] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 11:40:29.167595 19918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 FOLLOWER]: Leader election won for term 2
I20260620 11:40:29.167709 19918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 LEADER]: Becoming Leader. State: Replica: 22536e15c20e4537bb7ddfc8967e1a8f, State: Running, Role: LEADER
I20260620 11:40:29.167809 19918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:29.167872 19914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:29.168116 19919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } } }
I20260620 11:40:29.168208 19919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:29.168465 19920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22536e15c20e4537bb7ddfc8967e1a8f. Latest consensus state: current_term: 2 leader_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } } }
I20260620 11:40:29.168582 19920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:29.168725 19922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:40:29.169577 19922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:29.169687 19922 catalog_manager.cc:1259] Loaded cluster ID: a6684628ada543b7977a81979eeff5ac
I20260620 11:40:29.169726 19922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:40:29.170001 19922 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 11:40:29.175251 19922 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 11:40:29.178282 19922 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:40:29.178550 19922 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Loaded TSK: 0
I20260620 11:40:29.178797 19922 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 11:40:29.246961 19916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:29.247205 19916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:29.247236 19916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:29.248610 19916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:29.248674 19916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:29.248706 19916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:40:29.250567 19916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:40161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=35293
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:29.250905 19916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:29.251185 19916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:29.256963 19944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.256970 19941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.257011 19942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:29.257412 19916 server_base.cc:1034] running on GCE node
I20260620 11:40:29.257655 19916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:29.257999 19916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:29.259182 19916 hybrid_clock.cc:648] HybridClock initialized: now 1781955629259160 us; error 37 us; skew 500 ppm
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:29.262217 19916 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:40:29.262779 19916 webserver.cc:460] Webserver started at http://127.5.122.129:35293/ using document root <none> and password file <none>
I20260620 11:40:29.263054 19916 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:29.263145 19916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:29.264739 19916 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:40:29.265636 19951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.265877 19916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:29.266026 19916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "56e88bc305f44e5287a07699fc31edbe"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "439f8ef7456cd4b7cc51d5388d28c5ff"
server_key_iv: "13d914580e2d6c3a7a73c114cb1d9cb9"
server_key_version: "encryptionkey@0"
I20260620 11:40:29.266548 19916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:29.310318 19916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:29.312925 19916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:29.313095 19916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:29.313407 19916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:29.313987 19958 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 11:40:29.315584 19916 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 11:40:29.315645 19916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 11:40:29.315676 19916 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 11:40:29.316368 19916 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 11:40:29.316404 19916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 11:40:29.316524 19958 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap starting.
I20260620 11:40:29.323590 19916 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:40161
I20260620 11:40:29.323710 20065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:40161 every 8 connection(s)
I20260620 11:40:29.324059 19916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:40:29.327255 19958 log.cc:824] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:29.329509 19958 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:40:29.329818 19958 tablet_bootstrap.cc:492] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap complete.
I20260620 11:40:29.330904 19958 ts_tablet_manager.cc:1397] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.005s
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:29.332043 19958 raft_consensus.cc:348] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.332398 19958 raft_consensus.cc:729] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e88bc305f44e5287a07699fc31edbe, State: Initialized, Role: FOLLOWER
I20260620 11:40:29.332566 19958 consensus_queue.cc:260] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.332926 19958 ts_tablet_manager.cc:1428] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260620 11:40:29.333117 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 19916
I20260620 11:40:29.333463 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:41515
--local_ip_for_outbound_sockets=127.5.122.130
--tserver_master_addrs=127.5.122.190:39689
--webserver_port=33487
--webserver_interface=127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:29.336413 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.325002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:51751 (local address 127.5.122.190:39689)
0620 11:40:29.325809 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.325836 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.325868 (+ 32us) server_negotiation.cc:407] Connection header received
0620 11:40:29.326041 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.326048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.326100 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.326242 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:29.326917 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.327879 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.328944 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.329443 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.329575 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.331993 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:29.331998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:29.331998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:29.332012 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:29.334291 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.334832 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.334836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.334837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.334988 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.335305 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.335305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.335306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.335575 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:29.335642 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.336032 (+ 390us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.336154 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":105,"threads_started":1}
I20260620 11:40:29.336457 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.324881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:51751)
0620 11:40:29.325085 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.325132 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.325624 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.326276 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.326280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.326587 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:29.326784 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.326790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.328007 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.328008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.328821 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.328824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.329568 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.329569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.329718 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.330361 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:29.330370 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:29.331858 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:29.334403 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.334410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.334412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.334666 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.335070 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.335072 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.335072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.335235 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.335650 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:29.335656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:29.335861 (+ 205us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.335986 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.336184 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260620 11:40:29.337335 20066 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39689
I20260620 11:40:29.337455 20066 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:29.337661 20066 heartbeater.cc:511] Master 127.5.122.190:39689 requested a full tablet report, sending...
I20260620 11:40:29.338455 19878 ts_manager.cc:194] Registered new tserver with Master: 56e88bc305f44e5287a07699fc31edbe (127.5.122.129:40161)
I20260620 11:40:29.339574 19878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:51751
I20260620 11:40:29.340728 20066 heartbeater.cc:503] Master 127.5.122.190:39689 was elected leader, sending a full tablet report...
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:40:29.347934 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.339122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:40161 (local address 127.0.0.1:39762)
0620 11:40:29.339333 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.339373 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.339437 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.339634 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.339638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.340065 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:29.340132 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.340136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.341112 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.341114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.341790 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.341794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.342370 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.342371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.342460 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.343043 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:29.343048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:29.344765 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:29.346622 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.346631 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.346634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.346848 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.347086 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.347087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.347088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.347156 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.347508 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:29.347515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:29.347573 (+ 58us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.347680 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.347823 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":127,"threads_started":1}
I20260620 11:40:29.347926 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.339191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.5.122.129:40161)
0620 11:40:29.339401 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.339406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.339438 (+ 32us) server_negotiation.cc:407] Connection header received
0620 11:40:29.339524 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.339527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.339547 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.339622 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:29.340231 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.341028 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.341894 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.342285 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.342369 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.344867 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:29.344877 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:29.344877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:29.344885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:29.346528 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.346946 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.346950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.346950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.347013 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.347221 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.347221 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.347222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.347423 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:29.347544 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.347716 (+ 172us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.347820 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260620 11:40:29.348361 20000 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a961fcdb0384155909717cede49cf89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:28 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:39762
I20260620 11:40:29.348898 20075 tablet_replica.cc:330] stopping tablet replica
I20260620 11:40:29.349022 20075 raft_consensus.cc:2229] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:40:29.349097 20075 raft_consensus.cc:2258] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:29.349408 20075 ts_tablet_manager.cc:1905] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:29.351814 20075 ts_tablet_manager.cc:1918] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:29.351883 20075 log.cc:1196] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/0a961fcdb0384155909717cede49cf89
I20260620 11:40:29.352139 20075 ts_tablet_manager.cc:1939] T 0a961fcdb0384155909717cede49cf89 P 56e88bc305f44e5287a07699fc31edbe: Deleting consensus metadata
I20260620 11:40:29.352746 19863 catalog_manager.cc:4841] TS 56e88bc305f44e5287a07699fc31edbe (127.5.122.129:40161): tablet 0a961fcdb0384155909717cede49cf89 (table test-table [id=af4f6c6c760045d68faeef7c18361be1]) successfully deleted
W20260620 11:40:29.425418 20072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:29.425724 20072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:29.425772 20072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:29.427193 20072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:29.427289 20072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:29.427340 20072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:40:29.429015 20072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:41515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=33487
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:29.429335 20072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:29.429628 20072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:29.435652 20081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.435652 20080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.435878 20083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:29.435667 20072 server_base.cc:1034] running on GCE node
I20260620 11:40:29.436160 20072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:29.436514 20072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:29.437693 20072 hybrid_clock.cc:648] HybridClock initialized: now 1781955629437637 us; error 78 us; skew 500 ppm
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:29.440542 20072 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:40:29.441092 20072 webserver.cc:460] Webserver started at http://127.5.122.130:33487/ using document root <none> and password file <none>
I20260620 11:40:29.441363 20072 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:29.441460 20072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:29.443022 20072 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:29.443780 20090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.444006 20072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:29.444124 20072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "61f366cafe5c4f4fadc5e69121d4a6b8"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "27a704337b643ac65aa3040ebe54f505"
server_key_iv: "207a125ffaa1df701013b64234b42f0c"
server_key_version: "encryptionkey@0"
I20260620 11:40:29.444542 20072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:29.460206 20072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:29.462767 20072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:29.463029 20072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:29.463331 20072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:29.463676 20072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:29.463733 20072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.463786 20072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:29.463835 20072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.470470 20072 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:41515
I20260620 11:40:29.470515 20203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:41515 every 8 connection(s)
I20260620 11:40:29.471015 20072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:29.480094 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 20072
I20260620 11:40:29.480410 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:36641
--local_ip_for_outbound_sockets=127.5.122.131
--tserver_master_addrs=127.5.122.190:39689
--webserver_port=33255
--webserver_interface=127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:29.482072 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.472006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:37443 (local address 127.5.122.190:39689)
0620 11:40:29.472066 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.472069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.472284 (+ 215us) server_negotiation.cc:407] Connection header received
0620 11:40:29.472791 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.472792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.472808 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.472888 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:29.473890 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.474436 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.475565 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.475815 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.475891 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.478553 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:29.478555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:29.478555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:29.478563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:29.480293 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.480885 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.480887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.480887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.480954 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.481252 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.481253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.481253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.481428 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:29.481516 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.481839 (+ 323us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.481949 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 11:40:29.482332 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.471892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:37443)
0620 11:40:29.472135 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.472181 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.472699 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.472947 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.472951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.473560 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:29.473787 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.473792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.474544 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.474546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.475442 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.475445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.475912 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.475913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.476029 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.476606 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:29.476615 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:29.478406 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:29.480417 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.480427 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.480430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.480788 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.481035 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.481036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.481037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.481177 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.481508 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:29.481516 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:29.481690 (+ 174us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.481812 (+ 122us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.482036 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":115,"threads_started":1}
I20260620 11:40:29.483018 20204 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39689
I20260620 11:40:29.483158 20204 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:29.483376 20204 heartbeater.cc:511] Master 127.5.122.190:39689 requested a full tablet report, sending...
I20260620 11:40:29.483860 19878 ts_manager.cc:194] Registered new tserver with Master: 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515)
I20260620 11:40:29.484306 19878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:37443
W20260620 11:40:29.567351 20207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:29.567703 20207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:29.567749 20207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:29.569361 20207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:29.569437 20207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:29.569487 20207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:40:29.571264 20207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:36641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=33255
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:29.571565 20207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:29.571822 20207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:29.577298 20215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.577284 20212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.577474 20213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:29.577781 20207 server_base.cc:1034] running on GCE node
I20260620 11:40:29.578193 20207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:29.578521 20207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:29.579703 20207 hybrid_clock.cc:648] HybridClock initialized: now 1781955629579684 us; error 43 us; skew 500 ppm
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:29.582755 20207 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:40:29.583308 20207 webserver.cc:460] Webserver started at http://127.5.122.131:33255/ using document root <none> and password file <none>
I20260620 11:40:29.583576 20207 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:29.583673 20207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:29.585213 20207 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:29.585851 20222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.586072 20207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:29.586180 20207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "cc377929f0004133b7638929ef1d288a"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "5f3b8fb14734652d7b6de5bc34ca303b"
server_key_iv: "081e5e2abfe2c51478258dbc62dee1e6"
server_key_version: "encryptionkey@0"
I20260620 11:40:29.586588 20207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:29.597668 20207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:29.600407 20207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:29.600597 20207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:29.600876 20207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:29.601214 20207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:29.601270 20207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.601315 20207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:29.601346 20207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.607594 20207 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:36641
I20260620 11:40:29.607754 20335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:36641 every 8 connection(s)
I20260620 11:40:29.608078 20207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:29.616909 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 20207
I20260620 11:40:29.620271 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.609072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:46395 (local address 127.5.122.190:39689)
0620 11:40:29.609169 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.609172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.609371 (+ 199us) server_negotiation.cc:407] Connection header received
0620 11:40:29.610033 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.610034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.610049 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.610120 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:29.611254 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.611785 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.612928 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.613165 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.613252 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.616378 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:29.616380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:29.616380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:29.616391 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:29.618484 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.619079 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.619080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.619081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.619141 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.619524 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.619525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.619525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.619662 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:29.619748 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.620064 (+ 316us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.620159 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 11:40:29.620404 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.609016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:46395)
0620 11:40:29.609217 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.609258 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.609799 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.610211 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.610215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.610912 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:29.611133 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.611139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.611919 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.611920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.612800 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.612804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.613273 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.613274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.613405 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.614132 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:29.614142 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:29.616209 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:29.618592 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.618602 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.618606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.618991 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.619222 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:29.619222 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:29.619223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:29.619396 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:29.619746 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:29.619752 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:29.619921 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.620014 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.620157 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
I20260620 11:40:29.620893 20336 heartbeater.cc:347] Connected to a master server at 127.5.122.190:39689
I20260620 11:40:29.621018 20336 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:29.621212 20336 heartbeater.cc:511] Master 127.5.122.190:39689 requested a full tablet report, sending...
I20260620 11:40:29.621644 19878 ts_manager.cc:194] Registered new tserver with Master: cc377929f0004133b7638929ef1d288a (127.5.122.131:36641)
I20260620 11:40:29.622153 19878 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:46395
I20260620 11:40:29.625711 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.618271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.5.122.190:39689)
0620 11:40:29.618409 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.618411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.618425 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:40:29.618497 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.618498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.618511 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.618580 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:29.619400 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.620206 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.621591 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.621878 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.621998 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.623344 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:29.623345 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:29.623345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:29.623352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:29.624630 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.624969 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.624969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.624970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.625024 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:29.625206 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:29.625207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:29.625207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:29.625332 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:29.625483 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.625518 (+ 35us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.625589 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260620 11:40:29.626576 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:40:29.629251 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.626952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.5.122.190:39689)
0620 11:40:29.627009 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.627011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.627100 (+ 89us) server_negotiation.cc:407] Connection header received
0620 11:40:29.627165 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.627166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.627198 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.627259 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:40:29.627376 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.627844 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.628575 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.628766 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.628818 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.628975 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:40:29.629077 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:40:29.629143 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.629175 (+ 32us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.629197 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 11:40:29.629578 19878 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:40:29.631440 19878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:40:29.633816 20000 tablet_service.cc:1467] Processing CreateTablet for tablet e83652db2b7b424d92eaeec8fe85f255 (DEFAULT_TABLE table=test-table [id=fa9d55edfa54474580d03277f4f67d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:29.634111 20000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83652db2b7b424d92eaeec8fe85f255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:29.635823 19958 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap starting.
I20260620 11:40:29.636168 19958 tablet_bootstrap.cc:654] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:29.637136 19958 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: No bootstrap required, opened a new log
I20260620 11:40:29.637194 19958 ts_tablet_manager.cc:1397] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:29.637360 19958 raft_consensus.cc:348] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.637463 19958 raft_consensus.cc:374] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:29.637530 19958 raft_consensus.cc:729] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e88bc305f44e5287a07699fc31edbe, State: Initialized, Role: FOLLOWER
I20260620 11:40:29.637614 19958 consensus_queue.cc:260] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.637701 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.633520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:36641 (local address 127.0.0.1:46596)
0620 11:40:29.633576 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.633588 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.633628 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.634254 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.634255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.634257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:29.634321 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.634322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.635272 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.635273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.636424 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.636426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.637413 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.637414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.637499 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.637526 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.637564 (+ 38us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.637623 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 11:40:29.637722 19958 ts_tablet_manager.cc:1428] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.637863 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.633654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41515 (local address 127.0.0.1:37114)
0620 11:40:29.633790 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.633800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.633841 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.634318 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.634318 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.634320 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:29.634365 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.634367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.635604 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.635605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.636643 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.636645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.637623 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.637624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.637698 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.637702 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.637732 (+ 30us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.637788 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260620 11:40:29.638290 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.633894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.5.122.130:41515)
0620 11:40:29.634125 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.634130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.634142 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:40:29.634184 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.634186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.634201 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.634322 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:29.634425 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.635490 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.636732 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.637521 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.638076 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.638149 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.638176 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.638228 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":117,"threads_started":1}
I20260620 11:40:29.638294 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.633628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46596 (local address 127.5.122.131:36641)
0620 11:40:29.634044 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.634051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.634061 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:40:29.634102 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.634104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.634127 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.634397 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:29.634461 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.635157 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.636526 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.637308 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.637879 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.638131 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.638158 (+ 27us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.638207 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":110,"threads_started":1}
I20260620 11:40:29.639089 20138 tablet_service.cc:1467] Processing CreateTablet for tablet e83652db2b7b424d92eaeec8fe85f255 (DEFAULT_TABLE table=test-table [id=fa9d55edfa54474580d03277f4f67d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:29.639089 20270 tablet_service.cc:1467] Processing CreateTablet for tablet e83652db2b7b424d92eaeec8fe85f255 (DEFAULT_TABLE table=test-table [id=fa9d55edfa54474580d03277f4f67d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:29.639453 20270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83652db2b7b424d92eaeec8fe85f255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:29.639452 20138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e83652db2b7b424d92eaeec8fe85f255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:29.643215 20346 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Bootstrap starting.
I20260620 11:40:29.643975 20346 tablet_bootstrap.cc:654] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:29.644384 20346 log.cc:824] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:29.644692 20347 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Bootstrap starting.
I20260620 11:40:29.645239 20346 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: No bootstrap required, opened a new log
I20260620 11:40:29.645332 20347 tablet_bootstrap.cc:654] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:29.645327 20346 ts_tablet_manager.cc:1397] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:40:29.645885 20347 log.cc:824] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:29.646700 20347 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: No bootstrap required, opened a new log
I20260620 11:40:29.646783 20347 ts_tablet_manager.cc:1397] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:40:29.647529 20346 raft_consensus.cc:348] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.647755 20346 raft_consensus.cc:374] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:29.647797 20346 raft_consensus.cc:729] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc377929f0004133b7638929ef1d288a, State: Initialized, Role: FOLLOWER
I20260620 11:40:29.647930 20346 consensus_queue.cc:260] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.648238 20346 ts_tablet_manager.cc:1428] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:40:29.648267 20336 heartbeater.cc:503] Master 127.5.122.190:39689 was elected leader, sending a full tablet report...
I20260620 11:40:29.648809 20347 raft_consensus.cc:348] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.648950 20347 raft_consensus.cc:374] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:29.648974 20347 raft_consensus.cc:729] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f366cafe5c4f4fadc5e69121d4a6b8, State: Initialized, Role: FOLLOWER
I20260620 11:40:29.649117 20347 consensus_queue.cc:260] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.649451 20347 ts_tablet_manager.cc:1428] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:40:29.649525 20204 heartbeater.cc:503] Master 127.5.122.190:39689 was elected leader, sending a full tablet report...
W20260620 11:40:29.721690 20205 tablet.cc:2369] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:40:29.791679 20350 raft_consensus.cc:482] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:29.791821 20350 raft_consensus.cc:504] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.792155 20350 leader_election.cc:288] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515), 56e88bc305f44e5287a07699fc31edbe (127.5.122.129:40161)
I20260620 11:40:29.795146 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.792453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:32793 (local address 127.5.122.129:40161)
0620 11:40:29.792496 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.792498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.792509 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:40:29.792584 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.792585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.792615 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.792690 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:29.792865 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.793398 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.794371 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.794782 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.794882 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.794965 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.795033 (+ 68us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.795078 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:40:29.795146 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.792373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:40161 (local address 127.5.122.131:32793)
0620 11:40:29.792421 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.792435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.792508 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.792696 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.792697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.792698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:29.792775 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.792777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.793488 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.793489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.794258 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.794260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.794880 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.794881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.794959 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.794962 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.795021 (+ 59us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.795065 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260620 11:40:29.795234 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.792525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:59711 (local address 127.5.122.130:41515)
0620 11:40:29.792572 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.792574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.792654 (+ 80us) server_negotiation.cc:407] Connection header received
0620 11:40:29.792726 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.792727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.792736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.792809 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:29.793010 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.793485 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.794486 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.794867 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.794954 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.794963 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.795109 (+ 146us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.795145 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:40:29.795295 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.792373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41515 (local address 127.5.122.131:59711)
0620 11:40:29.792567 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.792579 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:29.792670 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:29.792812 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:29.792813 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:29.792814 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:29.792894 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.792896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.793579 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.793580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.794401 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:29.794403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.794965 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.794966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:29.795035 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.795038 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:40:29.795104 (+ 66us) client_negotiation.cc:240] Negotiation successful
0620 11:40:29.795142 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":28,"thread_start_us":97,"threads_started":1}
I20260620 11:40:29.795516 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83652db2b7b424d92eaeec8fe85f255" candidate_uuid: "cc377929f0004133b7638929ef1d288a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e88bc305f44e5287a07699fc31edbe" is_pre_election: true
I20260620 11:40:29.795665 20020 raft_consensus.cc:2454] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc377929f0004133b7638929ef1d288a in term 0.
I20260620 11:40:29.795900 20224 leader_election.cc:302] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe, cc377929f0004133b7638929ef1d288a; no voters:
I20260620 11:40:29.795915 20158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83652db2b7b424d92eaeec8fe85f255" candidate_uuid: "cc377929f0004133b7638929ef1d288a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" is_pre_election: true
I20260620 11:40:29.796056 20350 raft_consensus.cc:2781] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:40:29.796082 20158 raft_consensus.cc:2454] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc377929f0004133b7638929ef1d288a in term 0.
I20260620 11:40:29.796125 20350 raft_consensus.cc:482] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:40:29.796149 20350 raft_consensus.cc:3037] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:29.796972 20350 raft_consensus.cc:504] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.797120 20350 leader_election.cc:288] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [CANDIDATE]: Term 1 election: Requested vote from peers 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515), 56e88bc305f44e5287a07699fc31edbe (127.5.122.129:40161)
I20260620 11:40:29.797307 20158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83652db2b7b424d92eaeec8fe85f255" candidate_uuid: "cc377929f0004133b7638929ef1d288a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8"
I20260620 11:40:29.797353 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e83652db2b7b424d92eaeec8fe85f255" candidate_uuid: "cc377929f0004133b7638929ef1d288a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e88bc305f44e5287a07699fc31edbe"
I20260620 11:40:29.797384 20158 raft_consensus.cc:3037] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:29.797463 20020 raft_consensus.cc:3037] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:29.798204 20158 raft_consensus.cc:2454] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc377929f0004133b7638929ef1d288a in term 1.
I20260620 11:40:29.798319 20020 raft_consensus.cc:2454] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc377929f0004133b7638929ef1d288a in term 1.
I20260620 11:40:29.798379 20226 leader_election.cc:302] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61f366cafe5c4f4fadc5e69121d4a6b8, cc377929f0004133b7638929ef1d288a; no voters:
I20260620 11:40:29.798487 20350 raft_consensus.cc:2781] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:29.798640 20350 raft_consensus.cc:686] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 1 LEADER]: Becoming Leader. State: Replica: cc377929f0004133b7638929ef1d288a, State: Running, Role: LEADER
I20260620 11:40:29.798749 20350 consensus_queue.cc:237] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:29.799609 19878 catalog_manager.cc:5495] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a reported cstate change: term changed from 0 to 1, leader changed from <none> to cc377929f0004133b7638929ef1d288a (127.5.122.131). New cstate: current_term: 1 leader_uuid: "cc377929f0004133b7638929ef1d288a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } health_report { overall_health: UNKNOWN } } }
I20260620 11:40:29.822175 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.819885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46604 (local address 127.5.122.131:36641)
0620 11:40:29.819965 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:29.819967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:29.819975 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:40:29.820012 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:29.820013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:29.820032 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:29.820096 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:40:29.820253 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.820727 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.821424 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:29.821608 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:29.821659 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:29.821775 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:40:29.821878 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:40:29.822017 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:29.822070 (+ 53us) server_negotiation.cc:299] Negotiation successful
0620 11:40:29.822090 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 11:40:29.824097 20158 raft_consensus.cc:1261] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Refusing update from remote peer cc377929f0004133b7638929ef1d288a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:29.824134 20020 raft_consensus.cc:1261] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Refusing update from remote peer cc377929f0004133b7638929ef1d288a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:29.824460 20350 consensus_queue.cc:1035] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [LEADER]: Connected to new peer: Peer: permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:29.824635 20353 consensus_queue.cc:1035] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 11:40:29.824790 20067 tablet.cc:2369] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:40:29.827044 20358 mvcc.cc:205] Tried to move back new op lower bound from 7298890259756744704 to 7298890259656400896. Current Snapshot: MvccSnapshot[applied={T|T < 7298890259756744704}]
I20260620 11:40:29.829210 19877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:29.829330 19877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:29.830180 19877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Sending DeleteTablet for 3 replicas of tablet e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:29.830560 20000 tablet_service.cc:1514] Processing DeleteTablet for tablet e83652db2b7b424d92eaeec8fe85f255 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:29 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:39762
I20260620 11:40:29.830585 20138 tablet_service.cc:1514] Processing DeleteTablet for tablet e83652db2b7b424d92eaeec8fe85f255 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:29 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37114
I20260620 11:40:29.830616 20270 tablet_service.cc:1514] Processing DeleteTablet for tablet e83652db2b7b424d92eaeec8fe85f255 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:29 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:46596
I20260620 11:40:29.830662 20075 tablet_replica.cc:330] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: stopping tablet replica
I20260620 11:40:29.830672 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19916
I20260620 11:40:29.830739 20075 raft_consensus.cc:2229] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:40:29.830792 20075 raft_consensus.cc:2258] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:29.830873 20367 tablet_replica.cc:330] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: stopping tablet replica
I20260620 11:40:29.830971 20367 raft_consensus.cc:2229] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:40:29.831058 20367 raft_consensus.cc:2258] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:29.831593 20367 ts_tablet_manager.cc:1905] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:29.831907 20366 tablet_replica.cc:330] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: stopping tablet replica
I20260620 11:40:29.832062 20366 raft_consensus.cc:2229] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:40:29.832234 20366 raft_consensus.cc:2258] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:29.832803 20366 ts_tablet_manager.cc:1905] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:29.833597 20367 ts_tablet_manager.cc:1918] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:29.833681 20367 log.cc:1196] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:29.834036 20367 ts_tablet_manager.cc:1939] T e83652db2b7b424d92eaeec8fe85f255 P cc377929f0004133b7638929ef1d288a: Deleting consensus metadata
I20260620 11:40:29.834534 19865 catalog_manager.cc:4841] TS cc377929f0004133b7638929ef1d288a (127.5.122.131:36641): tablet e83652db2b7b424d92eaeec8fe85f255 (table test-table [id=fa9d55edfa54474580d03277f4f67d78]) successfully deleted
I20260620 11:40:29.834915 20366 ts_tablet_manager.cc:1918] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:29.835016 20366 log.cc:1196] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:29.835274 20366 ts_tablet_manager.cc:1939] T e83652db2b7b424d92eaeec8fe85f255 P 61f366cafe5c4f4fadc5e69121d4a6b8: Deleting consensus metadata
I20260620 11:40:29.835701 19865 catalog_manager.cc:4841] TS 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515): tablet e83652db2b7b424d92eaeec8fe85f255 (table test-table [id=fa9d55edfa54474580d03277f4f67d78]) successfully deleted
W20260620 11:40:29.838686 19863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:40161 (error 108)
I20260620 11:40:29.838963 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 20072
I20260620 11:40:29.839895 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:29.839750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:40161 (local address 127.0.0.1:39768)
0620 11:40:29.839806 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:29.839851 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:40161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 11:40:29.840094 19863 catalog_manager.cc:4584] TS 56e88bc305f44e5287a07699fc31edbe (127.5.122.129:40161): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e83652db2b7b424d92eaeec8fe85f255: Network error: Client connection negotiation failed: client connection to 127.5.122.129:40161: connect: Connection refused (error 111)
I20260620 11:40:29.845626 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 20207
I20260620 11:40:29.851048 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 19845
I20260620 11:40:29.856832 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39689
--webserver_interface=127.5.122.190
--webserver_port=43347
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:39689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:29.944464 20369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:29.944761 20369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:29.944801 20369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:29.946077 20369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:29.946164 20369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:29.946188 20369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:29.946208 20369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:29.946237 20369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:29.947877 20369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:39689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:39689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=43347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:29.948202 20369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:29.948458 20369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:29.954391 20375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.954365 20374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:29.954394 20377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:29.954820 20369 server_base.cc:1034] running on GCE node
I20260620 11:40:29.955281 20369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:29.955670 20369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:29.956828 20369 hybrid_clock.cc:648] HybridClock initialized: now 1781955629956796 us; error 45 us; skew 500 ppm
Jun 20 11:40:29 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955629, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:29.959649 20369 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:29.960175 20369 webserver.cc:460] Webserver started at http://127.5.122.190:43347/ using document root <none> and password file <none>
I20260620 11:40:29.960424 20369 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:29.960472 20369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:29.961903 20369 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:29.962752 20384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:29.963006 20369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:29.963104 20369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "22536e15c20e4537bb7ddfc8967e1a8f"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "41b76673e0b3073a659f53dce9fabfc8"
server_key_iv: "1ecb7a0e4dd79035245af55dee01211e"
server_key_version: "encryptionkey@0"
I20260620 11:40:29.963608 20369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:29.977125 20369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:29.979497 20369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:29.979665 20369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:29.984084 20369 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:39689
I20260620 11:40:29.984118 20436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:39689 every 8 connection(s)
I20260620 11:40:29.984526 20369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:29.986339 20437 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:40:29.987084 20437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Bootstrap starting.
I20260620 11:40:29.993302 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 20369
I20260620 11:40:29.993705 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:40161
--local_ip_for_outbound_sockets=127.5.122.129
--tserver_master_addrs=127.5.122.190:39689
--webserver_port=35293
--webserver_interface=127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:29.996647 20437 log.cc:824] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:29.999155 20437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:40:29.999496 20437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f: Bootstrap complete.
I20260620 11:40:30.001868 20437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:30.002231 20437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22536e15c20e4537bb7ddfc8967e1a8f, State: Initialized, Role: FOLLOWER
I20260620 11:40:30.002463 20437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:30.002568 20437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:40:30.002624 20437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:40:30.002676 20437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 2 FOLLOWER]: Advancing to term 3
I20260620 11:40:30.003813 20437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:30.003968 20437 leader_election.cc:302] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22536e15c20e4537bb7ddfc8967e1a8f; no voters:
I20260620 11:40:30.004194 20437 leader_election.cc:288] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 11:40:30.004241 20441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 3 FOLLOWER]: Leader election won for term 3
I20260620 11:40:30.004359 20441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [term 3 LEADER]: Becoming Leader. State: Replica: 22536e15c20e4537bb7ddfc8967e1a8f, State: Running, Role: LEADER
I20260620 11:40:30.004459 20441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } }
I20260620 11:40:30.004505 20437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:30.004758 20442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } } }
I20260620 11:40:30.004858 20442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:30.005030 20443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22536e15c20e4537bb7ddfc8967e1a8f. Latest consensus state: current_term: 3 leader_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22536e15c20e4537bb7ddfc8967e1a8f" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 39689 } } }
I20260620 11:40:30.005087 20443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22536e15c20e4537bb7ddfc8967e1a8f [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:30.005236 20446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:40:30.006146 20446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:30.006247 20446 catalog_manager.cc:1259] Loaded cluster ID: a6684628ada543b7977a81979eeff5ac
I20260620 11:40:30.006475 20446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 11:40:30.006700 20446 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f807b26b9ad google::LogMessage::Fail() at ??:0
@ 0x7f807b26e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f807b26b4de google::LogMessage::Flush() at ??:0
@ 0x7f807b26ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f808318c0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f80831958b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f807d1390d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f807d12e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f807df926db start_thread at ??:0
@ 0x7f8079d6961f clone at ??:0
W20260620 11:40:30.094537 20439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:30.094801 20439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:30.094852 20439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:30.096104 20439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:30.096187 20439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:30.096235 20439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:40:30.097951 20439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:40161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=35293
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:30.098361 20439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:30.098632 20439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:30.105381 20473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:30.105400 20439 server_base.cc:1034] running on GCE node
W20260620 11:40:30.105376 20476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:30.105505 20474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:30.105921 20439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:30.106273 20439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:30.107501 20439 hybrid_clock.cc:648] HybridClock initialized: now 1781955630107493 us; error 45 us; skew 500 ppm
Jun 20 11:40:30 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955630, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:30.111156 20439 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:40:30.111825 20439 webserver.cc:460] Webserver started at http://127.5.122.129:35293/ using document root <none> and password file <none>
I20260620 11:40:30.112129 20439 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:30.112206 20439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:30.114356 20439 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:40:30.115432 20483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:30.115706 20439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:30.115856 20439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "56e88bc305f44e5287a07699fc31edbe"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "439f8ef7456cd4b7cc51d5388d28c5ff"
server_key_iv: "13d914580e2d6c3a7a73c114cb1d9cb9"
server_key_version: "encryptionkey@0"
I20260620 11:40:30.116437 20439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:30.136323 20439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:30.139221 20439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:30.139394 20439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:30.139660 20439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:30.140188 20490 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 11:40:30.141810 20439 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 11:40:30.141887 20439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 11:40:30.141928 20439 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 11:40:30.142901 20439 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 11:40:30.142953 20439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 11:40:30.143062 20490 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap starting.
I20260620 11:40:30.150632 20439 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:40161
I20260620 11:40:30.150875 20597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:40161 every 8 connection(s)
I20260620 11:40:30.151199 20439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:40:30.154175 20490 log.cc:824] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:30.157236 20490 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:40:30.157598 20490 tablet_bootstrap.cc:492] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Bootstrap complete.
I20260620 11:40:30.158784 20490 ts_tablet_manager.cc:1397] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260620 11:40:30.160024 20490 raft_consensus.cc:348] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:30.160347 20490 raft_consensus.cc:729] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e88bc305f44e5287a07699fc31edbe, State: Initialized, Role: FOLLOWER
I20260620 11:40:30.160523 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 20439
I20260620 11:40:30.160527 20490 consensus_queue.cc:260] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:30.160835 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:41515
--local_ip_for_outbound_sockets=127.5.122.130
--tserver_master_addrs=127.5.122.190:39689
--webserver_port=33487
--webserver_interface=127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:40:30.160871 20490 ts_tablet_manager.cc:1428] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260620 11:40:30.235549 20600 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:40:30.152080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:45593)
0620 11:40:30.152338 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.152395 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:30.153193 (+ 798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:30.235300 (+ 82107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":108,"threads_started":1}
W20260620 11:40:30.235849 20598 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:39689 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.190:39689: Client connection negotiation failed: client connection to 127.5.122.190:39689: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 11:40:30.236265 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.236135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:50993)
0620 11:40:30.236195 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.236244 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 11:40:30.236699 20600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.236594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:44729)
0620 11:40:30.236645 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.236681 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 11:40:30.236819 20598 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 11:40:30.248947 20603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:30.249225 20603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:30.249271 20603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:30.250865 20603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:30.250952 20603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:30.251003 20603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:40:30.252662 20603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:41515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=33487
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:30.252987 20603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:30.253249 20603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:30.258946 20609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:30.258930 20608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:30.258930 20611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:30.259089 20603 server_base.cc:1034] running on GCE node
I20260620 11:40:30.259512 20603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:30.259768 20603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:30.260941 20603 hybrid_clock.cc:648] HybridClock initialized: now 1781955630260909 us; error 49 us; skew 500 ppm
Jun 20 11:40:30 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955630, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:30.263883 20603 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:40:30.264442 20603 webserver.cc:460] Webserver started at http://127.5.122.130:33487/ using document root <none> and password file <none>
I20260620 11:40:30.264664 20603 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:30.264709 20603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:30.266242 20603 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:30.267007 20618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:30.267269 20603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:30.267372 20603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "61f366cafe5c4f4fadc5e69121d4a6b8"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "27a704337b643ac65aa3040ebe54f505"
server_key_iv: "207a125ffaa1df701013b64234b42f0c"
server_key_version: "encryptionkey@0"
I20260620 11:40:30.267809 20603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:30.276921 20603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:30.279642 20603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:30.279876 20603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:30.280149 20603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:30.280489 20603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:30.280525 20603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:30.280550 20603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:30.280593 20603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:30.286857 20603 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:41515
I20260620 11:40:30.286976 20731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:41515 every 8 connection(s)
I20260620 11:40:30.287328 20603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:40:30.288813 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.288241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:60315)
0620 11:40:30.288527 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.288610 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":88,"threads_started":1}
W20260620 11:40:30.289340 20732 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:39689 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.190:39689: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
I20260620 11:40:30.289685 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.289579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:46087)
0620 11:40:30.289625 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.289666 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260620 11:40:30.290212 20734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.290113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:45137)
0620 11:40:30.290162 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.290192 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 11:40:30.290369 20732 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 11:40:30.296622 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 20603
I20260620 11:40:30.296988 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:36641
--local_ip_for_outbound_sockets=127.5.122.131
--tserver_master_addrs=127.5.122.190:39689
--webserver_port=33255
--webserver_interface=127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:30.383623 20735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:30.383914 20735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:30.383957 20735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:30.385236 20735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:30.385324 20735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:30.385371 20735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:40:30.387080 20735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:36641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=33255
--tserver_master_addrs=127.5.122.190:39689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:30.387393 20735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:30.387663 20735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:30.393064 20741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:30.393106 20740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:30.393123 20735 server_base.cc:1034] running on GCE node
W20260620 11:40:30.393085 20743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:30.393632 20735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:30.393949 20735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:30.395184 20735 hybrid_clock.cc:648] HybridClock initialized: now 1781955630395168 us; error 42 us; skew 500 ppm
Jun 20 11:40:30 dist-test-slave-szdr krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955630, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:30.398277 20735 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:40:30.398877 20735 webserver.cc:460] Webserver started at http://127.5.122.131:33255/ using document root <none> and password file <none>
I20260620 11:40:30.399140 20735 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:30.399217 20735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:30.400818 20735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:30.401597 20750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:30.401954 20735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:30.402146 20735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "cc377929f0004133b7638929ef1d288a"
format_stamp: "Formatted at 2026-06-20 11:40:28 on dist-test-slave-szdr"
server_key: "5f3b8fb14734652d7b6de5bc34ca303b"
server_key_iv: "081e5e2abfe2c51478258dbc62dee1e6"
server_key_version: "encryptionkey@0"
I20260620 11:40:30.402567 20735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:30.416183 20735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:30.418985 20735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:30.419179 20735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:30.419464 20735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:30.419814 20735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:30.419872 20735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:30.419909 20735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:30.419942 20735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:30.426177 20735 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:36641
I20260620 11:40:30.426231 20863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:36641 every 8 connection(s)
I20260620 11:40:30.426611 20735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:40:30.427798 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.427324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:40005)
0620 11:40:30.427493 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.427617 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260620 11:40:30.428300 20864 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:39689 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.190:39689: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
I20260620 11:40:30.428632 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.428533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:50299)
0620 11:40:30.428581 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.428612 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 11:40:30.428938 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:30.428846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:46945)
0620 11:40:30.428902 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:30.428924 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 11:40:30.429057 20864 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 11:40:30.432740 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 20735
I20260620 11:40:31.237551 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:31.237289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:48819)
0620 11:40:31.237436 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:31.237485 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260620 11:40:31.291106 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:31.290818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:40783)
0620 11:40:31.290993 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:31.291041 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
I20260620 11:40:31.429757 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:31.429452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:33573)
0620 11:40:31.429606 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:31.429652 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260620 11:40:31.812343 20871 raft_consensus.cc:482] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:31.812515 20871 raft_consensus.cc:504] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:31.812829 20871 leader_election.cc:288] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc377929f0004133b7638929ef1d288a (127.5.122.131:36641), 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515)
Jun 20 11:40:31 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955630, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.131@KRBTEST.COM
Jun 20 11:40:31 dist-test-slave-szdr krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955630, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.130@KRBTEST.COM
I20260620 11:40:31.827149 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:31.813421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:52535 (local address 127.5.122.130:41515)
0620 11:40:31.813565 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:31.813572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:31.813590 (+ 18us) server_negotiation.cc:407] Connection header received
0620 11:40:31.814189 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:31.814192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:31.814231 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:31.814400 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:31.815619 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.817121 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.818832 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.819527 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.819655 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:31.823428 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:31.823433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:31.823433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:31.823446 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:31.825505 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:31.825981 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:31.825985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:31.825985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:31.826098 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:31.826300 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:31.826300 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:31.826301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:31.826577 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:31.826657 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:31.826870 (+ 213us) server_negotiation.cc:299] Negotiation successful
0620 11:40:31.827009 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260620 11:40:31.827132 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:31.813050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:36641 (local address 127.5.122.129:50281)
0620 11:40:31.813180 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:31.813194 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:31.813254 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:31.814513 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:31.814517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:31.815129 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:31.815452 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:31.815462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.817275 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.817277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:31.818715 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:31.818720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.819671 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.819673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:31.819849 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:31.820628 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:31.820641 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:31.822925 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:31.825085 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:31.825094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:31.825097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:31.825487 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:31.825805 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:31.825806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:31.825806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:31.826003 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:31.826522 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:31.826529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:31.826683 (+ 154us) client_negotiation.cc:769] Sending connection context
0620 11:40:31.826791 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 11:40:31.827031 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":97,"thread_start_us":91,"threads_started":1}
I20260620 11:40:31.827173 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:31.813171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:50281 (local address 127.5.122.131:36641)
0620 11:40:31.813389 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:31.813405 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:31.813445 (+ 40us) server_negotiation.cc:407] Connection header received
0620 11:40:31.814146 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:31.814150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:31.814226 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:31.814400 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:31.815619 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.817121 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.818832 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.819527 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.819654 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:31.823135 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:31.823141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:31.823141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:31.823155 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:31.824973 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:31.825605 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:31.825609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:31.825610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:31.825727 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:31.826077 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:31.826078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:31.826078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:31.826452 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:31.826514 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:31.826885 (+ 371us) server_negotiation.cc:299] Negotiation successful
0620 11:40:31.827049 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":55,"threads_started":1}
I20260620 11:40:31.827256 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:31.813276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41515 (local address 127.5.122.129:52535)
0620 11:40:31.813383 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:31.813393 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:31.813440 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:31.814513 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:31.814517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:31.815129 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:31.815452 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:31.815462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.817279 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.817280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:31.818719 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:31.818720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:31.819673 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:31.819673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:31.819849 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:31.820628 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:31.820640 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:31.823310 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:31.825596 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:31.825598 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:31.825598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:31.825835 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:31.826171 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:31.826172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:31.826173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:31.826234 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:31.826654 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:31.826655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:31.826696 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 11:40:31.826810 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 11:40:31.827116 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":229,"thread_start_us":57,"threads_started":1}
W20260620 11:40:31.828018 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc377929f0004133b7638929ef1d288a (127.5.122.131:36641): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
W20260620 11:40:31.828653 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:31.828723 20487 leader_election.cc:302] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe; no voters: 61f366cafe5c4f4fadc5e69121d4a6b8, cc377929f0004133b7638929ef1d288a
I20260620 11:40:31.828859 20871 raft_consensus.cc:2726] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:40:32.238276 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:32.238132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:57447)
0620 11:40:32.238193 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:32.238233 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 11:40:32.292022 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:32.291743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:48641)
0620 11:40:32.291912 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:32.291969 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260620 11:40:32.430500 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:32.430276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:39231)
0620 11:40:32.430413 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:32.430454 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260620 11:40:33.239209 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:33.238883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:35623)
0620 11:40:33.239076 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:33.239126 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260620 11:40:33.292851 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:33.292616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:54229)
0620 11:40:33.292754 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:33.292803 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260620 11:40:33.431420 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:33.431154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:57161)
0620 11:40:33.431317 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:33.431366 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260620 11:40:33.536649 20881 raft_consensus.cc:482] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:33.536831 20881 raft_consensus.cc:504] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:33.537060 20881 leader_election.cc:288] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc377929f0004133b7638929ef1d288a (127.5.122.131:36641), 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515)
W20260620 11:40:33.537925 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc377929f0004133b7638929ef1d288a (127.5.122.131:36641): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
W20260620 11:40:33.538103 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:33.538126 20487 leader_election.cc:302] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe; no voters: 61f366cafe5c4f4fadc5e69121d4a6b8, cc377929f0004133b7638929ef1d288a
I20260620 11:40:33.538261 20881 raft_consensus.cc:2726] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:40:34.240016 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:34.239777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:37831)
0620 11:40:34.239934 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:34.239970 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260620 11:40:34.293673 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:34.293452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:48805)
0620 11:40:34.293589 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:34.293624 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260620 11:40:34.432478 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:34.432240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:51087)
0620 11:40:34.432396 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:34.432427 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260620 11:40:35.240967 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:35.240649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:42801)
0620 11:40:35.240860 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:35.240916 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260620 11:40:35.294564 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:35.294320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:55043)
0620 11:40:35.294469 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:35.294517 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260620 11:40:35.433359 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:35.433119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:52125)
0620 11:40:35.433263 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:35.433312 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260620 11:40:35.475024 20888 raft_consensus.cc:482] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:35.475142 20888 raft_consensus.cc:504] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:35.475365 20888 leader_election.cc:288] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc377929f0004133b7638929ef1d288a (127.5.122.131:36641), 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515)
W20260620 11:40:35.475898 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc377929f0004133b7638929ef1d288a (127.5.122.131:36641): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
W20260620 11:40:35.475992 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:35.476042 20487 leader_election.cc:302] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe; no voters: 61f366cafe5c4f4fadc5e69121d4a6b8, cc377929f0004133b7638929ef1d288a
I20260620 11:40:35.476145 20888 raft_consensus.cc:2726] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:40:36.241919 20889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:36.241603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:48307)
0620 11:40:36.241813 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:36.241866 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260620 11:40:36.295455 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:36.295221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:46833)
0620 11:40:36.295361 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:36.295405 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260620 11:40:36.434230 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:36.433935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:52779)
0620 11:40:36.434132 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:36.434180 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":112,"threads_started":1}
I20260620 11:40:37.242916 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:37.242646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:44871)
0620 11:40:37.242808 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:37.242857 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
I20260620 11:40:37.296250 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:37.295984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:55931)
0620 11:40:37.296148 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:37.296199 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260620 11:40:37.434979 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:37.434746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:57431)
0620 11:40:37.434886 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:37.434929 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260620 11:40:37.631963 20895 raft_consensus.cc:482] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:37.632062 20895 raft_consensus.cc:504] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:37.632189 20895 leader_election.cc:288] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc377929f0004133b7638929ef1d288a (127.5.122.131:36641), 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515)
W20260620 11:40:37.632627 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
W20260620 11:40:37.632709 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc377929f0004133b7638929ef1d288a (127.5.122.131:36641): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:37.632735 20487 leader_election.cc:302] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe; no voters: 61f366cafe5c4f4fadc5e69121d4a6b8, cc377929f0004133b7638929ef1d288a
I20260620 11:40:37.632833 20895 raft_consensus.cc:2726] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:40:38.243930 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:38.243648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:59651)
0620 11:40:38.243843 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:38.243882 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":125,"threads_started":1}
I20260620 11:40:38.297343 20897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:38.297037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:35651)
0620 11:40:38.297230 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:38.297276 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":77,"threads_started":1}
I20260620 11:40:38.435909 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:38.435647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:34253)
0620 11:40:38.435811 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:38.435860 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260620 11:40:39.244867 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:39.244553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:43747)
0620 11:40:39.244757 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:39.244812 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
I20260620 11:40:39.298297 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:39.298018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:51323)
0620 11:40:39.298196 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:39.298247 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":122,"threads_started":1}
I20260620 11:40:39.436796 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:39.436517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:43223)
0620 11:40:39.436696 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:39.436746 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260620 11:40:40.245901 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:40.245619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:57369)
0620 11:40:40.245798 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:40.245848 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
I20260620 11:40:40.299216 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:40.298923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:36423)
0620 11:40:40.299118 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:40.299169 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":82,"threads_started":1}
I20260620 11:40:40.437693 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:40.437438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:47755)
0620 11:40:40.437592 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:40.437641 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260620 11:40:41.246975 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:41.246702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:53895)
0620 11:40:41.246877 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:41.246926 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260620 11:40:41.300117 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:41.299841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:37977)
0620 11:40:41.300019 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:41.300069 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":68,"threads_started":1}
I20260620 11:40:41.438697 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:41.438331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:56567)
0620 11:40:41.438580 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:41.438640 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":83,"threads_started":1}
I20260620 11:40:42.247920 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:42.247627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:47489)
0620 11:40:42.247816 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:42.247859 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260620 11:40:42.301196 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:42.300796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:40301)
0620 11:40:42.301017 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:42.301119 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260620 11:40:42.439631 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:42.439338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:38001)
0620 11:40:42.439520 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:42.439572 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
I20260620 11:40:43.248764 20912 raft_consensus.cc:482] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:43.248874 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:43.248539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:38903)
0620 11:40:43.248691 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:43.248742 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260620 11:40:43.248907 20912 raft_consensus.cc:504] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc377929f0004133b7638929ef1d288a" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36641 } } peers { permanent_uuid: "61f366cafe5c4f4fadc5e69121d4a6b8" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41515 } } peers { permanent_uuid: "56e88bc305f44e5287a07699fc31edbe" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 40161 } }
I20260620 11:40:43.249082 20912 leader_election.cc:288] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc377929f0004133b7638929ef1d288a (127.5.122.131:36641), 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515)
W20260620 11:40:43.249488 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 61f366cafe5c4f4fadc5e69121d4a6b8 (127.5.122.130:41515): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
W20260620 11:40:43.249552 20487 leader_election.cc:341] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc377929f0004133b7638929ef1d288a (127.5.122.131:36641): Not found: Tablet not found: e83652db2b7b424d92eaeec8fe85f255
I20260620 11:40:43.249568 20487 leader_election.cc:302] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56e88bc305f44e5287a07699fc31edbe; no voters: 61f366cafe5c4f4fadc5e69121d4a6b8, cc377929f0004133b7638929ef1d288a
I20260620 11:40:43.249637 20912 raft_consensus.cc:2726] T e83652db2b7b424d92eaeec8fe85f255 P 56e88bc305f44e5287a07699fc31edbe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:40:43.302165 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:43.301822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:52299)
0620 11:40:43.302061 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:43.302113 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":71,"threads_started":1}
I20260620 11:40:43.440651 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:43.440354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:35227)
0620 11:40:43.440547 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:43.440597 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260620 11:40:44.249895 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:44.249603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:45427)
0620 11:40:44.249790 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:44.249845 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":123,"threads_started":1}
I20260620 11:40:44.303079 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:44.302812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:35323)
0620 11:40:44.302979 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:44.303029 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260620 11:40:44.441542 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:44.441261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:55265)
0620 11:40:44.441428 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:44.441485 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
I20260620 11:40:45.251014 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:45.250751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.129:39101)
0620 11:40:45.250915 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:45.250970 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260620 11:40:45.304061 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:45.303800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.130:52841)
0620 11:40:45.303965 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:45.304014 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":71,"threads_started":1}
I20260620 11:40:45.442667 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:45.442358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:39689 (local address 127.5.122.131:34003)
0620 11:40:45.442566 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:45.442618 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:39689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":68,"threads_started":1}
I20260620 11:40:45.443801 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 20439
I20260620 11:40:45.449069 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 20603
I20260620 11:40:45.453722 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 20735
2026-06-20T11:40:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18475 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:40:45 dist-test-slave-szdr krb5kdc[20923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:40:45 dist-test-slave-szdr krb5kdc[20923](info): set up 2 sockets
Jun 20 11:40:45 dist-test-slave-szdr krb5kdc[20923](info): commencing operation
krb5kdc: starting...
W20260620 11:40:46.506299 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:40:46 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:40:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:40:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:40:46.637603 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35853
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:35853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:46.724133 20936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:46.724426 20936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:46.724466 20936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:46.725773 20936 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 11:40:46.725868 20936 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 11:40:46.725893 20936 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 11:40:46.725914 20936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:40:46.725946 20936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:46.726034 20936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:46.726068 20936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:40:46.726099 20936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:40:46.727880 20936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:35853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:46.728226 20936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:46.728574 20936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:46.734248 20944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:46.734241 20942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:46.734277 20936 server_base.cc:1034] running on GCE node
W20260620 11:40:46.734241 20941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:46.734846 20936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:46.735188 20936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:46.736368 20936 hybrid_clock.cc:648] HybridClock initialized: now 1781955646736356 us; error 51 us; skew 500 ppm
Jun 20 11:40:46 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955646, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:46.739275 20936 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:40:46.739773 20936 webserver.cc:460] Webserver started at http://127.5.122.190:45883/ using document root <none> and password file <none>
I20260620 11:40:46.740046 20936 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:46.740131 20936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:46.740295 20936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:46.741452 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "21fe7010f3d74440813517381035cd63"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "e4a465061f28c1795991d4c3f56313a9"
server_key_iv: "b4143d4a9546e142e5498d3b6517c5a8"
server_key_version: "encryptionkey@0"
I20260620 11:40:46.741852 20936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "21fe7010f3d74440813517381035cd63"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "e4a465061f28c1795991d4c3f56313a9"
server_key_iv: "b4143d4a9546e142e5498d3b6517c5a8"
server_key_version: "encryptionkey@0"
I20260620 11:40:46.743462 20936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:40:46.744422 20951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:46.744665 20936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:46.744794 20936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "21fe7010f3d74440813517381035cd63"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "e4a465061f28c1795991d4c3f56313a9"
server_key_iv: "b4143d4a9546e142e5498d3b6517c5a8"
server_key_version: "encryptionkey@0"
I20260620 11:40:46.744884 20936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:46.771806 20936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:46.774288 20936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:46.774500 20936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:46.778962 20936 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:35853
I20260620 11:40:46.779011 21003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:35853 every 8 connection(s)
I20260620 11:40:46.779395 20936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:40:46.780305 21004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:46.782665 21004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Bootstrap starting.
I20260620 11:40:46.782969 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 20936
I20260620 11:40:46.783093 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:40:46.783279 5610 external_mini_cluster.cc:1428] Setting key ce8e4f2c3502eb5373bbfee9df493983
I20260620 11:40:46.783452 21004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:46.783783 21004 log.cc:824] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:46.784682 21004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: No bootstrap required, opened a new log
I20260620 11:40:46.787045 21004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:40:46.787232 21004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:46.787268 21004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21fe7010f3d74440813517381035cd63, State: Initialized, Role: FOLLOWER
I20260620 11:40:46.787396 21004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:40:46.787474 21004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:40:46.787503 21004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:40:46.787549 21004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:46.788276 21004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:40:46.788370 21004 leader_election.cc:302] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21fe7010f3d74440813517381035cd63; no voters:
I20260620 11:40:46.788591 21004 leader_election.cc:288] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:40:46.788617 21009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:46.788734 21009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 LEADER]: Becoming Leader. State: Replica: 21fe7010f3d74440813517381035cd63, State: Running, Role: LEADER
I20260620 11:40:46.788821 21009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:40:46.788889 21004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:40:46.789162 21009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21fe7010f3d74440813517381035cd63. Latest consensus state: current_term: 1 leader_uuid: "21fe7010f3d74440813517381035cd63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } } }
I20260620 11:40:46.789253 21009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:46.789175 21011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21fe7010f3d74440813517381035cd63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } } }
I20260620 11:40:46.789453 21011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: This master's current role is: LEADER
I20260620 11:40:46.790692 21013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Jun 20 11:40:46 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:46.791131 21013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:40:46.793208 21013 catalog_manager.cc:1347] Generated new cluster ID: 8bdb81490e8045e78ae8b1ed1cb87b46
I20260620 11:40:46.793308 21013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:40:46.795724 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:46.783870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.5.122.190:35853)
0620 11:40:46.784085 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:46.784094 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:46.784139 (+ 45us) server_negotiation.cc:407] Connection header received
0620 11:40:46.784875 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:46.784885 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:46.784978 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:46.785174 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:46.786030 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:46.786950 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:46.788040 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:46.788610 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:46.789106 (+ 496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:46.791736 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:46.791740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:46.791741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:46.791753 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:46.793999 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:46.794444 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:46.794447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:46.794448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:46.794555 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:46.794796 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:46.794796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:46.794797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:46.795092 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:46.795229 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:46.795387 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 11:40:46.795521 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
I20260620 11:40:46.806555 21013 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:40:46.807144 21013 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:40:46.814697 21013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Generated new TSK 0
I20260620 11:40:46.814905 21013 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:40:46.850209 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35853
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:46.956703 21030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:46.956956 21030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:46.956987 21030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:46.958423 21030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:46.958484 21030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:46.958516 21030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:40:46.960282 21030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:46.960601 21030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:46.960894 21030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:46.966506 21036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:46.966506 21035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:46.966506 21038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:46.966565 21030 server_base.cc:1034] running on GCE node
I20260620 11:40:46.966956 21030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:46.967273 21030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:46.968544 21030 hybrid_clock.cc:648] HybridClock initialized: now 1781955646968526 us; error 31 us; skew 500 ppm
Jun 20 11:40:46 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955646, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:46.971735 21030 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:40:46.972370 21030 webserver.cc:460] Webserver started at http://127.5.122.129:43997/ using document root <none> and password file <none>
I20260620 11:40:46.972723 21030 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:46.972808 21030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:46.972980 21030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:46.974306 21030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "1e5518978f45465bb8ca9249ee7f1f96"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "b33cc4edebb9a2763937bdf438b54b4e"
server_key_iv: "907a52017e9de550ee54c5d1c54ff5bb"
server_key_version: "encryptionkey@0"
I20260620 11:40:46.974733 21030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "1e5518978f45465bb8ca9249ee7f1f96"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "b33cc4edebb9a2763937bdf438b54b4e"
server_key_iv: "907a52017e9de550ee54c5d1c54ff5bb"
server_key_version: "encryptionkey@0"
I20260620 11:40:46.976572 21030 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 11:40:46.977563 21045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:46.977823 21030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:46.977989 21030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "1e5518978f45465bb8ca9249ee7f1f96"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "b33cc4edebb9a2763937bdf438b54b4e"
server_key_iv: "907a52017e9de550ee54c5d1c54ff5bb"
server_key_version: "encryptionkey@0"
I20260620 11:40:46.978101 21030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:46.989210 21030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:46.991966 21030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:46.992173 21030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:46.992456 21030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:46.992821 21030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:46.992887 21030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:46.992928 21030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:46.992978 21030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:46.999665 21030 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:46371
I20260620 11:40:46.999730 21158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:46371 every 8 connection(s)
I20260620 11:40:47.000152 21030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:40:47.006989 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21030
I20260620 11:40:47.007088 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:40:47.007295 5610 external_mini_cluster.cc:1428] Setting key 9916eec7c193885c131d97de129f6164
Jun 20 11:40:47 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955646, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:47.012107 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.001120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:35341 (local address 127.5.122.190:35853)
0620 11:40:47.001186 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.001188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.001336 (+ 148us) server_negotiation.cc:407] Connection header received
0620 11:40:47.001898 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.001900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.001912 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.002069 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:47.003170 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.003882 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.004969 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.005198 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.005274 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.008158 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:47.008159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:47.008160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:47.008167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:47.010095 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.010755 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.010757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.010757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.010819 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.011219 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.011221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.011221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.011352 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:47.011434 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.011929 (+ 495us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.012024 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:40:47.012392 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.000979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:35341)
0620 11:40:47.001182 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.001230 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.001767 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.002208 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.002212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.002814 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:47.003020 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.003026 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.003988 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.003989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.004841 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.004844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.005292 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.005293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.005398 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.006053 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:47.006063 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:47.008036 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:47.010213 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:47.010221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:47.010223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:47.010600 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:47.010908 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:47.010909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:47.010910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:47.011123 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:47.011524 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:47.011533 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:47.011741 (+ 208us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.011872 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.012087 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":115,"threads_started":1}
I20260620 11:40:47.013034 21159 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35853
I20260620 11:40:47.013166 21159 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:47.013373 21159 heartbeater.cc:511] Master 127.5.122.190:35853 requested a full tablet report, sending...
I20260620 11:40:47.014086 20968 ts_manager.cc:194] Registered new tserver with Master: 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371)
I20260620 11:40:47.014729 20968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:35341
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:40:47.037559 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35853
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:47.122959 21164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:47.123202 21164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:47.123231 21164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:47.124586 21164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:47.124646 21164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:47.124676 21164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:40:47.126461 21164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:47.126788 21164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:47.127068 21164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:47.132602 21170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:47.132633 21172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:47.132618 21169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:47.132833 21164 server_base.cc:1034] running on GCE node
I20260620 11:40:47.133109 21164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:47.133417 21164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:47.134572 21164 hybrid_clock.cc:648] HybridClock initialized: now 1781955647134537 us; error 50 us; skew 500 ppm
Jun 20 11:40:47 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:47.137431 21164 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:40:47.138049 21164 webserver.cc:460] Webserver started at http://127.5.122.130:44209/ using document root <none> and password file <none>
I20260620 11:40:47.138302 21164 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:47.138373 21164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:47.138527 21164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:47.139665 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "2cad626b94154223aff38c15075f65e2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "73e24b3c860d5348907e046f295944d3"
server_key_iv: "2bb0046a2e33324fbecf33d913f8e5e7"
server_key_version: "encryptionkey@0"
I20260620 11:40:47.140064 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "2cad626b94154223aff38c15075f65e2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "73e24b3c860d5348907e046f295944d3"
server_key_iv: "2bb0046a2e33324fbecf33d913f8e5e7"
server_key_version: "encryptionkey@0"
I20260620 11:40:47.141551 21164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:40:47.142463 21179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:47.142714 21164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:40:47.142817 21164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "2cad626b94154223aff38c15075f65e2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "73e24b3c860d5348907e046f295944d3"
server_key_iv: "2bb0046a2e33324fbecf33d913f8e5e7"
server_key_version: "encryptionkey@0"
I20260620 11:40:47.142910 21164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:47.168547 21164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:47.171078 21164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:47.171288 21164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:47.171559 21164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:47.171911 21164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:47.171972 21164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:47.172017 21164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:47.172068 21164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:47.178666 21164 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:41987
I20260620 11:40:47.178707 21292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:41987 every 8 connection(s)
I20260620 11:40:47.179070 21164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:40:47.183490 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21164
I20260620 11:40:47.183614 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:40:47.183804 5610 external_mini_cluster.cc:1428] Setting key 59c86116ac277962ba542e4503736ef9
Jun 20 11:40:47 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:47.191546 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.179926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:36109 (local address 127.5.122.190:35853)
0620 11:40:47.179992 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.179995 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.180212 (+ 217us) server_negotiation.cc:407] Connection header received
0620 11:40:47.180697 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.180698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.180710 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.180779 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:47.182076 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.182603 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.184136 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.184427 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.184522 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.187824 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:47.187826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:47.187826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:47.187836 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:47.189804 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.190404 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.190406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.190407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.190481 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.190790 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.190791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.190792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.190973 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:47.191052 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.191337 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.191436 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 11:40:47.191591 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.179835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:36109)
0620 11:40:47.180046 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.180090 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.180595 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.180889 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.180893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.181654 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:47.181886 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.181892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.182752 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.182753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.184015 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.184026 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.184614 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.184616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.184803 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.185574 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:47.185589 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:47.187703 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:47.189941 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:47.189950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:47.189953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:47.190259 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:47.190562 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:47.190563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:47.190564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:47.190702 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:47.191045 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:47.191050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:47.191181 (+ 131us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.191265 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.191401 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260620 11:40:47.192178 21293 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35853
I20260620 11:40:47.192318 21293 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:47.192523 21293 heartbeater.cc:511] Master 127.5.122.190:35853 requested a full tablet report, sending...
I20260620 11:40:47.192979 20968 ts_manager.cc:194] Registered new tserver with Master: 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
I20260620 11:40:47.193384 20968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:36109
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:40:47.216701 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35853
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:40:47.307765 21298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:40:47.308029 21298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:40:47.308061 21298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:40:47.309408 21298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:40:47.309458 21298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:40:47.309494 21298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:40:47.311373 21298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:40:47.311717 21298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:40:47.312131 21298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:40:47.318020 21304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:47.318053 21303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:40:47.318020 21306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:40:47.318217 21298 server_base.cc:1034] running on GCE node
I20260620 11:40:47.318513 21298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:40:47.318796 21298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:40:47.319984 21298 hybrid_clock.cc:648] HybridClock initialized: now 1781955647319985 us; error 62 us; skew 500 ppm
Jun 20 11:40:47 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:40:47.322870 21298 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:40:47.323441 21298 webserver.cc:460] Webserver started at http://127.5.122.131:34055/ using document root <none> and password file <none>
I20260620 11:40:47.323702 21298 fs_manager.cc:359] Metadata directory not provided
I20260620 11:40:47.323776 21298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:40:47.323926 21298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:40:47.325101 21298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "dbd879990fd9428e81e95344e52f6ee2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "626b0cca3c7b918801f018ce1c03de9d"
server_key_iv: "23f94f02be6209cd2b21a84105bc0afb"
server_key_version: "encryptionkey@0"
I20260620 11:40:47.325491 21298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "dbd879990fd9428e81e95344e52f6ee2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "626b0cca3c7b918801f018ce1c03de9d"
server_key_iv: "23f94f02be6209cd2b21a84105bc0afb"
server_key_version: "encryptionkey@0"
I20260620 11:40:47.327034 21298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:40:47.327975 21313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:47.328244 21298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:40:47.328370 21298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "dbd879990fd9428e81e95344e52f6ee2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "626b0cca3c7b918801f018ce1c03de9d"
server_key_iv: "23f94f02be6209cd2b21a84105bc0afb"
server_key_version: "encryptionkey@0"
I20260620 11:40:47.328459 21298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:40:47.348920 21298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:40:47.351637 21298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:40:47.351763 21298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:40:47.352134 21298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:40:47.352560 21298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:40:47.352600 21298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:47.352625 21298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:40:47.352638 21298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:40:47.358824 21298 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:46761
I20260620 11:40:47.358870 21426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:46761 every 8 connection(s)
I20260620 11:40:47.359216 21298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:40:47.362304 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21298
I20260620 11:40:47.362380 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:40:47.362565 5610 external_mini_cluster.cc:1428] Setting key 484126e01651bba22bda32e43629f4b7
Jun 20 11:40:47 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955647, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:40:47.369949 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.359999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:50599 (local address 127.5.122.190:35853)
0620 11:40:47.360067 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.360069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.360288 (+ 219us) server_negotiation.cc:407] Connection header received
0620 11:40:47.360802 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.360803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.360816 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.360876 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:47.361872 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.362467 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.364019 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.364244 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.364319 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.367025 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:47.367027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:47.367028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:47.367036 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:47.368499 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.368942 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.368944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.368944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.369009 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.369279 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.369280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.369280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.369419 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:47.369481 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.369780 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.369875 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 11:40:47.370234 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.359955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:50599)
0620 11:40:47.360145 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.360185 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.360695 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.360953 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.360957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.361561 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:40:47.361771 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.361777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.362674 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.362675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.363902 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.363906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.364418 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.364420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.364541 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.365085 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:40:47.365095 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:40:47.366905 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:40:47.368595 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:47.368602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:47.368604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:47.368849 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:47.369085 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:40:47.369086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:40:47.369086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:40:47.369208 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:40:47.369506 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:40:47.369511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:40:47.369645 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.369741 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.369879 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":122,"threads_started":1}
I20260620 11:40:47.370873 21427 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35853
I20260620 11:40:47.371018 21427 heartbeater.cc:464] Registering TS with master...
I20260620 11:40:47.371234 21427 heartbeater.cc:511] Master 127.5.122.190:35853 requested a full tablet report, sending...
I20260620 11:40:47.371682 20968 ts_manager.cc:194] Registered new tserver with Master: dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761)
I20260620 11:40:47.372079 20968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:50599
I20260620 11:40:47.374307 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:40:47.382627 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.375729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.5.122.190:35853)
0620 11:40:47.375775 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.375777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.375912 (+ 135us) server_negotiation.cc:407] Connection header received
0620 11:40:47.375967 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.375967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.375976 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.376024 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:40:47.376668 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.377168 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.378038 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.378227 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.378286 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.379571 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:40:47.379573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:40:47.379573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:40:47.379580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:40:47.381061 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.381453 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.381454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.381455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.381510 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:40:47.381730 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:40:47.381731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:40:47.381731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:40:47.381875 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:40:47.382039 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.382284 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.382375 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:40:47.384600 20968 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:40:47.386482 20968 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:40:47.394950 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.389424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:46761 (local address 127.0.0.1:41780)
0620 11:40:47.389562 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.389574 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.389623 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.390376 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.390380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.390383 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:47.390471 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.390474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.391991 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.391992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.393248 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.393252 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.394559 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.394560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.394677 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.394696 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.394770 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.394841 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260620 11:40:47.395047 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.389347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46371 (local address 127.0.0.1:48402)
0620 11:40:47.389484 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.389545 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.389623 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.390568 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.390569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.390572 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:47.390630 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.390632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.391727 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.391729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.393256 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.393257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.394793 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.394794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.394850 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.394853 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.394883 (+ 30us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.394934 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260620 11:40:47.395256 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.389685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.5.122.131:46761)
0620 11:40:47.389884 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.389888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.389914 (+ 26us) server_negotiation.cc:407] Connection header received
0620 11:40:47.389961 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.389963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.390001 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.390084 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:47.390572 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.391876 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.393347 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.394446 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.394542 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.394626 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.395114 (+ 488us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.395178 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":117,"threads_started":1}
I20260620 11:40:47.395463 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.390114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48402 (local address 127.5.122.129:46371)
0620 11:40:47.390343 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.390348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.390395 (+ 47us) server_negotiation.cc:407] Connection header received
0620 11:40:47.390440 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.390442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.390458 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.390707 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:47.390786 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.391622 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.394065 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.394683 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.395197 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.395291 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.395320 (+ 29us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.395377 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
I20260620 11:40:47.395815 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.389584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41987 (local address 127.0.0.1:44550)
0620 11:40:47.389717 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.389729 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.389784 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.390625 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.390626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.390627 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:47.390661 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.390663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.392144 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.392146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.393155 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.393158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.394867 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.394868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.394954 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.394964 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.395221 (+ 257us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.395271 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260620 11:40:47.395972 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.390140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44550 (local address 127.5.122.130:41987)
0620 11:40:47.390343 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.390348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.390359 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:40:47.390449 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.390451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.390466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.390543 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:47.390754 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.392014 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.393274 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.394444 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.394761 (+ 317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.394838 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.395388 (+ 550us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.395445 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260620 11:40:47.396170 21361 tablet_service.cc:1467] Processing CreateTablet for tablet d4f204bc0e124a23a70a992e4d44b7f6 (DEFAULT_TABLE table=test-table [id=754e718cee6d4b9090ba419157dcebed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:47.396250 21093 tablet_service.cc:1467] Processing CreateTablet for tablet d4f204bc0e124a23a70a992e4d44b7f6 (DEFAULT_TABLE table=test-table [id=754e718cee6d4b9090ba419157dcebed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:47.396561 21361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4f204bc0e124a23a70a992e4d44b7f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:47.396641 21093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4f204bc0e124a23a70a992e4d44b7f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:47.398766 21227 tablet_service.cc:1467] Processing CreateTablet for tablet d4f204bc0e124a23a70a992e4d44b7f6 (DEFAULT_TABLE table=test-table [id=754e718cee6d4b9090ba419157dcebed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:40:47.399183 21227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4f204bc0e124a23a70a992e4d44b7f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:40:47.400111 21444 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Bootstrap starting.
I20260620 11:40:47.400518 21445 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap starting.
I20260620 11:40:47.401099 21444 tablet_bootstrap.cc:654] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:47.401188 21445 tablet_bootstrap.cc:654] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:47.401854 21445 log.cc:824] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:47.401854 21444 log.cc:824] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:47.402943 21444 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: No bootstrap required, opened a new log
I20260620 11:40:47.402943 21445 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: No bootstrap required, opened a new log
I20260620 11:40:47.403023 21444 ts_tablet_manager.cc:1397] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:40:47.403023 21445 ts_tablet_manager.cc:1397] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:40:47.405144 21445 raft_consensus.cc:348] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.405148 21444 raft_consensus.cc:348] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.405351 21445 raft_consensus.cc:374] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:47.405351 21444 raft_consensus.cc:374] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:47.405390 21445 raft_consensus.cc:729] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5518978f45465bb8ca9249ee7f1f96, State: Initialized, Role: FOLLOWER
I20260620 11:40:47.405390 21444 raft_consensus.cc:729] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbd879990fd9428e81e95344e52f6ee2, State: Initialized, Role: FOLLOWER
I20260620 11:40:47.405521 21445 consensus_queue.cc:260] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.405581 21444 consensus_queue.cc:260] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.402640 21448 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Bootstrap starting.
I20260620 11:40:47.405855 21445 ts_tablet_manager.cc:1428] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:40:47.405854 21444 ts_tablet_manager.cc:1428] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:40:47.406023 21427 heartbeater.cc:503] Master 127.5.122.190:35853 was elected leader, sending a full tablet report...
I20260620 11:40:47.406076 21159 heartbeater.cc:503] Master 127.5.122.190:35853 was elected leader, sending a full tablet report...
I20260620 11:40:47.406585 21448 tablet_bootstrap.cc:654] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Neither blocks nor log segments found. Creating new log.
I20260620 11:40:47.407351 21448 log.cc:824] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:40:47.408185 21448 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: No bootstrap required, opened a new log
I20260620 11:40:47.408288 21448 ts_tablet_manager.cc:1397] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260620 11:40:47.409766 21448 raft_consensus.cc:348] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.409961 21448 raft_consensus.cc:374] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:40:47.410035 21448 raft_consensus.cc:729] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cad626b94154223aff38c15075f65e2, State: Initialized, Role: FOLLOWER
I20260620 11:40:47.410168 21448 consensus_queue.cc:260] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.410494 21448 ts_tablet_manager.cc:1428] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 11:40:47.410555 21293 heartbeater.cc:503] Master 127.5.122.190:35853 was elected leader, sending a full tablet report...
W20260620 11:40:47.429800 21294 tablet.cc:2369] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:40:47.501016 21160 tablet.cc:2369] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:40:47.526176 21450 raft_consensus.cc:482] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:40:47.526309 21450 raft_consensus.cc:504] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.526736 21450 leader_election.cc:288] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371), 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
I20260620 11:40:47.529841 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.527068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41987 (local address 127.5.122.131:51729)
0620 11:40:47.527265 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.527277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.527348 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.527481 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.527482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.527484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:47.527548 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.527549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.528197 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.528198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.528960 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.528963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.529546 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.529547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.529621 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.529632 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.529674 (+ 42us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.529706 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":112,"threads_started":1}
I20260620 11:40:47.529841 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.527164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:51729 (local address 127.5.122.130:41987)
0620 11:40:47.527213 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.527215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.527351 (+ 136us) server_negotiation.cc:407] Connection header received
0620 11:40:47.527406 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.527407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.527418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.527468 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:47.527611 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.528088 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.529033 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.529454 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.529513 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.529520 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.529701 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.529753 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 11:40:47.529886 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.527050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:37315 (local address 127.5.122.129:46371)
0620 11:40:47.527107 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.527109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.527121 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:40:47.527158 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.527159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.527167 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.527273 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:40:47.527463 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.528023 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.529104 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.529473 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.529573 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.529581 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.529761 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.529810 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 11:40:47.529847 21429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.526915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46371 (local address 127.5.122.131:37315)
0620 11:40:47.526976 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.526990 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:40:47.527038 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:40:47.527277 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:40:47.527278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:40:47.527280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:40:47.527370 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.527372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.528115 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.528116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.528988 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:40:47.528991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.529592 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.529593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:40:47.529676 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.529679 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:40:47.529737 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 11:40:47.529773 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 11:40:47.530488 21247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f204bc0e124a23a70a992e4d44b7f6" candidate_uuid: "dbd879990fd9428e81e95344e52f6ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cad626b94154223aff38c15075f65e2" is_pre_election: true
I20260620 11:40:47.530504 21113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f204bc0e124a23a70a992e4d44b7f6" candidate_uuid: "dbd879990fd9428e81e95344e52f6ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5518978f45465bb8ca9249ee7f1f96" is_pre_election: true
I20260620 11:40:47.530648 21247 raft_consensus.cc:2454] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbd879990fd9428e81e95344e52f6ee2 in term 0.
I20260620 11:40:47.530660 21113 raft_consensus.cc:2454] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbd879990fd9428e81e95344e52f6ee2 in term 0.
I20260620 11:40:47.530929 21317 leader_election.cc:302] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e5518978f45465bb8ca9249ee7f1f96, dbd879990fd9428e81e95344e52f6ee2; no voters:
I20260620 11:40:47.531062 21450 raft_consensus.cc:2781] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:40:47.531126 21450 raft_consensus.cc:482] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:40:47.531152 21450 raft_consensus.cc:3037] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:47.531960 21450 raft_consensus.cc:504] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.532105 21450 leader_election.cc:288] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [CANDIDATE]: Term 1 election: Requested vote from peers 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371), 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
I20260620 11:40:47.532279 21113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f204bc0e124a23a70a992e4d44b7f6" candidate_uuid: "dbd879990fd9428e81e95344e52f6ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5518978f45465bb8ca9249ee7f1f96"
I20260620 11:40:47.532312 21247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f204bc0e124a23a70a992e4d44b7f6" candidate_uuid: "dbd879990fd9428e81e95344e52f6ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cad626b94154223aff38c15075f65e2"
I20260620 11:40:47.532354 21113 raft_consensus.cc:3037] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:47.532425 21247 raft_consensus.cc:3037] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:40:47.533171 21113 raft_consensus.cc:2454] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbd879990fd9428e81e95344e52f6ee2 in term 1.
I20260620 11:40:47.533313 21247 raft_consensus.cc:2454] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbd879990fd9428e81e95344e52f6ee2 in term 1.
I20260620 11:40:47.533362 21317 leader_election.cc:302] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e5518978f45465bb8ca9249ee7f1f96, dbd879990fd9428e81e95344e52f6ee2; no voters:
I20260620 11:40:47.533456 21450 raft_consensus.cc:2781] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:40:47.533627 21450 raft_consensus.cc:686] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 1 LEADER]: Becoming Leader. State: Replica: dbd879990fd9428e81e95344e52f6ee2, State: Running, Role: LEADER
I20260620 11:40:47.533735 21450 consensus_queue.cc:237] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:40:47.534647 20968 catalog_manager.cc:5495] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "dbd879990fd9428e81e95344e52f6ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } health_report { overall_health: HEALTHY } } }
I20260620 11:40:47.541427 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.539204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.5.122.131:46761)
0620 11:40:47.539289 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 11:40:47.539291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:40:47.539300 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:40:47.539328 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:40:47.539329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:40:47.539345 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:40:47.539411 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:40:47.539530 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.540015 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.540758 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:40:47.540959 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:40:47.541042 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:40:47.541172 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:40:47.541267 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:40:47.541324 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 11:40:47.541352 (+ 28us) server_negotiation.cc:299] Negotiation successful
0620 11:40:47.541370 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 11:40:47.543371 21247 raft_consensus.cc:1261] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 1 FOLLOWER]: Refusing update from remote peer dbd879990fd9428e81e95344e52f6ee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:47.543431 21113 raft_consensus.cc:1261] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Refusing update from remote peer dbd879990fd9428e81e95344e52f6ee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:40:47.543780 21450 consensus_queue.cc:1035] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:47.543926 21454 consensus_queue.cc:1035] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:40:47.546444 21457 mvcc.cc:205] Tried to move back new op lower bound from 7298890332334895104 to 7298890332299243520. Current Snapshot: MvccSnapshot[applied={T|T < 7298890332334895104}]
I20260620 11:40:47.549170 20968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:47.549284 20968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:40:47.550184 20968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Sending DeleteTablet for 3 replicas of tablet d4f204bc0e124a23a70a992e4d44b7f6
I20260620 11:40:47.550612 21361 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f204bc0e124a23a70a992e4d44b7f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:47 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:41780
I20260620 11:40:47.550688 21093 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f204bc0e124a23a70a992e4d44b7f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:47 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:48402
I20260620 11:40:47.550690 21227 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f204bc0e124a23a70a992e4d44b7f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:47 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:44550
I20260620 11:40:47.550917 21467 tablet_replica.cc:330] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: stopping tablet replica
I20260620 11:40:47.550938 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 21030
I20260620 11:40:47.554191 21468 tablet_replica.cc:330] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: stopping tablet replica
I20260620 11:40:47.554334 21468 raft_consensus.cc:2229] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:40:47.554458 21468 raft_consensus.cc:2258] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:47.554646 21466 tablet_replica.cc:330] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: stopping tablet replica
I20260620 11:40:47.554793 21466 raft_consensus.cc:2229] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:40:47.554961 21466 raft_consensus.cc:2258] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:40:47.555061 21468 ts_tablet_manager.cc:1905] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:40:47.555477 21466 ts_tablet_manager.cc:1905] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 11:40:47.556785 20955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:46371 (error 108)
I20260620 11:40:47.557029 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 21164
I20260620 11:40:47.559274 21466 ts_tablet_manager.cc:1918] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:40:47.559372 21466 log.cc:1196] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/d4f204bc0e124a23a70a992e4d44b7f6
I20260620 11:40:47.559690 21466 ts_tablet_manager.cc:1939] T d4f204bc0e124a23a70a992e4d44b7f6 P dbd879990fd9428e81e95344e52f6ee2: Deleting consensus metadata
I20260620 11:40:47.560441 20954 catalog_manager.cc:4841] TS dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761): tablet d4f204bc0e124a23a70a992e4d44b7f6 (table test-table [id=754e718cee6d4b9090ba419157dcebed]) successfully deleted
I20260620 11:40:47.563663 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 21298
I20260620 11:40:47.563810 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.563651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46371 (local address 127.0.0.1:48408)
0620 11:40:47.563722 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.563773 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 11:40:47.563905 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:40:47.563739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41987 (local address 127.0.0.1:44554)
0620 11:40:47.563878 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 11:40:47.563894 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.130:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260620 11:40:47.569209 20955 catalog_manager.cc:4584] TS 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4f204bc0e124a23a70a992e4d44b7f6: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46371: connect: Connection refused (error 111)
I20260620 11:40:47.569339 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 20936
I20260620 11:41:07.575162 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35853
--webserver_interface=127.5.122.190
--webserver_port=45883
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:35853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:07.662056 21470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:07.662281 21470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:07.662302 21470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:07.663529 21470 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 11:41:07.663574 21470 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 11:41:07.663589 21470 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 11:41:07.663599 21470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:07.663612 21470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:07.663625 21470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:07.663683 21470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:07.663702 21470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:07.665427 21470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:35853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=45883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:07.665721 21470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:07.666052 21470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:07.675611 21479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:07.675586 21476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:07.675587 21477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:07.675774 21470 server_base.cc:1034] running on GCE node
I20260620 11:41:07.676244 21470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:07.676544 21470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:07.677780 21470 hybrid_clock.cc:648] HybridClock initialized: now 1781955667677748 us; error 46 us; skew 500 ppm
Jun 20 11:41:07 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955667, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:07.684965 21470 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:41:07.685628 21470 webserver.cc:460] Webserver started at http://127.5.122.190:45883/ using document root <none> and password file <none>
I20260620 11:41:07.685882 21470 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:07.685945 21470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:07.687659 21470 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:07.688465 21486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:07.688711 21470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:07.688799 21470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "21fe7010f3d74440813517381035cd63"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "e4a465061f28c1795991d4c3f56313a9"
server_key_iv: "b4143d4a9546e142e5498d3b6517c5a8"
server_key_version: "encryptionkey@0"
I20260620 11:41:07.689302 21470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:07.703485 21470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:07.706185 21470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:07.706342 21470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:07.710923 21470 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:35853
I20260620 11:41:07.710960 21538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:35853 every 8 connection(s)
I20260620 11:41:07.711441 21470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:07.713096 21539 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:41:07.713821 21539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Bootstrap starting.
I20260620 11:41:07.721359 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21470
I20260620 11:41:07.721685 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:46371
--local_ip_for_outbound_sockets=127.5.122.129
--tserver_master_addrs=127.5.122.190:35853
--webserver_port=43997
--webserver_interface=127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:41:07.722744 21539 log.cc:824] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:07.724956 21539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:41:07.725334 21539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Bootstrap complete.
I20260620 11:41:07.727771 21539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:07.728037 21539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21fe7010f3d74440813517381035cd63, State: Initialized, Role: FOLLOWER
I20260620 11:41:07.728253 21539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:07.728350 21539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:07.728410 21539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:07.728458 21539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 1 FOLLOWER]: Advancing to term 2
I20260620 11:41:07.729523 21539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:07.729645 21539 leader_election.cc:302] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21fe7010f3d74440813517381035cd63; no voters:
I20260620 11:41:07.729859 21539 leader_election.cc:288] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 11:41:07.729902 21543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 11:41:07.730072 21543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 LEADER]: Becoming Leader. State: Replica: 21fe7010f3d74440813517381035cd63, State: Running, Role: LEADER
I20260620 11:41:07.730187 21539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:07.730227 21543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:07.730717 21544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "21fe7010f3d74440813517381035cd63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } } }
I20260620 11:41:07.730734 21545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21fe7010f3d74440813517381035cd63. Latest consensus state: current_term: 2 leader_uuid: "21fe7010f3d74440813517381035cd63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } } }
I20260620 11:41:07.730826 21544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:07.730858 21545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:07.731287 21552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:07.732162 21552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:07.732275 21552 catalog_manager.cc:1259] Loaded cluster ID: 8bdb81490e8045e78ae8b1ed1cb87b46
I20260620 11:41:07.732353 21552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:07.733723 21552 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:07.733909 21552 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Loaded TSK: 0
I20260620 11:41:07.747690 21552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Generated new TSK 1
I20260620 11:41:07.747804 21552 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 11:41:07.826438 21541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:07.826797 21541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:07.826848 21541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:07.829658 21541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:07.829763 21541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:07.829823 21541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:07.833179 21541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:46371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=43997
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:07.833551 21541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:07.833876 21541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:07.840018 21566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:07.840080 21567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:07.840512 21569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:07.843690 21541 server_base.cc:1034] running on GCE node
I20260620 11:41:07.844146 21541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:07.844506 21541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:07.845679 21541 hybrid_clock.cc:648] HybridClock initialized: now 1781955667845654 us; error 38 us; skew 500 ppm
Jun 20 11:41:07 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955667, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:07.848809 21541 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:41:07.849386 21541 webserver.cc:460] Webserver started at http://127.5.122.129:43997/ using document root <none> and password file <none>
I20260620 11:41:07.849674 21541 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:07.849736 21541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:07.851428 21541 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:07.852300 21576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:07.852515 21541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:07.852603 21541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "1e5518978f45465bb8ca9249ee7f1f96"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "b33cc4edebb9a2763937bdf438b54b4e"
server_key_iv: "907a52017e9de550ee54c5d1c54ff5bb"
server_key_version: "encryptionkey@0"
I20260620 11:41:07.853091 21541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:07.873006 21541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:07.875787 21541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:07.875948 21541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:07.876240 21541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:07.876730 21583 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 11:41:07.878412 21541 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 11:41:07.878504 21541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 11:41:07.878541 21541 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 11:41:07.879449 21541 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 11:41:07.879505 21541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260620 11:41:07.879595 21583 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap starting.
I20260620 11:41:07.887667 21541 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:46371
I20260620 11:41:07.887945 21690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:46371 every 8 connection(s)
I20260620 11:41:07.888166 21541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:41:07.888739 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21541
I20260620 11:41:07.889052 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:41987
--local_ip_for_outbound_sockets=127.5.122.130
--tserver_master_addrs=127.5.122.190:35853
--webserver_port=44209
--webserver_interface=127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:41:07.891608 21583 log.cc:824] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:07.894888 21583 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:41:07.895263 21583 tablet_bootstrap.cc:492] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap complete.
I20260620 11:41:07.896627 21583 ts_tablet_manager.cc:1397] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260620 11:41:07.898691 21583 raft_consensus.cc:348] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:41:07.899147 21583 raft_consensus.cc:729] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5518978f45465bb8ca9249ee7f1f96, State: Initialized, Role: FOLLOWER
I20260620 11:41:07.899369 21583 consensus_queue.cc:260] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } }
I20260620 11:41:07.899752 21583 ts_tablet_manager.cc:1428] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
Jun 20 11:41:07 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955667, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:07.906360 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:07.891249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:34053 (local address 127.5.122.190:35853)
0620 11:41:07.892881 (+ 1632us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:07.892889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:07.892920 (+ 31us) server_negotiation.cc:407] Connection header received
0620 11:41:07.893105 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:07.893110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:07.893158 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:07.893303 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:07.894226 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.895797 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.897356 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.897938 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.898076 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:07.901528 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:07.901536 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:07.901536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:07.901555 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:07.904004 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:07.904728 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:07.904736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:07.904736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:07.904966 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:07.905305 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:07.905305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:07.905306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:07.905563 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:07.905636 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:07.906065 (+ 429us) server_negotiation.cc:299] Negotiation successful
0620 11:41:07.906197 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1582,"thread_start_us":83,"threads_started":1}
I20260620 11:41:07.906420 21694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:07.891169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:34053)
0620 11:41:07.891943 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:07.891993 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:07.892686 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:07.893335 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:07.893339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:07.893761 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:07.894079 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:07.894087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.895932 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.895934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:07.897248 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:07.897252 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.898207 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.898208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:07.898344 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:07.899017 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:07.899029 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:07.901366 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:07.904146 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:07.904163 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:07.904167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:07.904551 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:07.905057 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:07.905058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:07.905059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:07.905229 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:07.905637 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:07.905643 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:07.905830 (+ 187us) client_negotiation.cc:769] Sending connection context
0620 11:41:07.905977 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 11:41:07.906196 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":108,"threads_started":1}
I20260620 11:41:07.907228 21691 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35853
I20260620 11:41:07.907352 21691 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:07.907559 21691 heartbeater.cc:511] Master 127.5.122.190:35853 requested a full tablet report, sending...
I20260620 11:41:07.908447 21503 ts_manager.cc:194] Registered new tserver with Master: 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371)
I20260620 11:41:07.909803 21503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:34053
I20260620 11:41:07.911072 21691 heartbeater.cc:503] Master 127.5.122.190:35853 was elected leader, sending a full tablet report...
Jun 20 11:41:07 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955667, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.129@KRBTEST.COM
I20260620 11:41:07.919198 21698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:07.909281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46371 (local address 127.0.0.1:35752)
0620 11:41:07.909498 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:07.909544 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:07.909617 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:07.910678 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:07.910685 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:07.910977 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:07.911034 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:07.911044 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.912136 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.912137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:07.912736 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:07.912739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.913261 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.913262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:07.913351 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:07.913921 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:07.913926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:07.915660 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:07.917722 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:07.917734 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:07.917736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:07.918085 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:07.918345 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:07.918346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:07.918347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:07.918417 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:07.918747 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:07.918756 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:07.918819 (+ 63us) client_negotiation.cc:769] Sending connection context
0620 11:41:07.918922 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 11:41:07.919094 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":120,"threads_started":1}
I20260620 11:41:07.919199 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:07.910296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.5.122.129:46371)
0620 11:41:07.910484 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:07.910489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:07.910501 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:41:07.910551 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:07.910553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:07.910566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:07.910660 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:07.911182 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.912041 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.912838 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:07.913172 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:07.913261 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:07.915779 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:07.915793 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:07.915793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:07.915802 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:07.917585 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:07.918186 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:07.918191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:07.918191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:07.918266 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:07.918483 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:07.918484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:07.918484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:07.918671 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:07.918770 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:07.918963 (+ 193us) server_negotiation.cc:299] Negotiation successful
0620 11:41:07.919087 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
I20260620 11:41:07.919770 21625 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f204bc0e124a23a70a992e4d44b7f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:40:47 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:35752
I20260620 11:41:07.920462 21700 tablet_replica.cc:330] stopping tablet replica
I20260620 11:41:07.920601 21700 raft_consensus.cc:2229] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:07.920682 21700 raft_consensus.cc:2258] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:07.920998 21700 ts_tablet_manager.cc:1905] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:07.923454 21700 ts_tablet_manager.cc:1918] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:07.923563 21700 log.cc:1196] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal/wals/d4f204bc0e124a23a70a992e4d44b7f6
I20260620 11:41:07.923794 21700 ts_tablet_manager.cc:1939] T d4f204bc0e124a23a70a992e4d44b7f6 P 1e5518978f45465bb8ca9249ee7f1f96: Deleting consensus metadata
I20260620 11:41:07.924443 21490 catalog_manager.cc:4841] TS 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371): tablet d4f204bc0e124a23a70a992e4d44b7f6 (table test-table [id=754e718cee6d4b9090ba419157dcebed]) successfully deleted
W20260620 11:41:07.986374 21693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:07.986642 21693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:07.986676 21693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:07.988039 21693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:07.988116 21693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:07.988149 21693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:07.989845 21693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:41987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=44209
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:07.990187 21693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:07.990417 21693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:07.996381 21705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:07.996404 21708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:07.996438 21693 server_base.cc:1034] running on GCE node
W20260620 11:41:07.996405 21706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:07.996871 21693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:07.997197 21693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:07.998370 21693 hybrid_clock.cc:648] HybridClock initialized: now 1781955667998341 us; error 40 us; skew 500 ppm
Jun 20 11:41:07 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955667, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:08.001441 21693 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:41:08.002063 21693 webserver.cc:460] Webserver started at http://127.5.122.130:44209/ using document root <none> and password file <none>
I20260620 11:41:08.002341 21693 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:08.002442 21693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:08.003959 21693 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 11:41:08.004807 21715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:08.005034 21693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:08.005155 21693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "2cad626b94154223aff38c15075f65e2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "73e24b3c860d5348907e046f295944d3"
server_key_iv: "2bb0046a2e33324fbecf33d913f8e5e7"
server_key_version: "encryptionkey@0"
I20260620 11:41:08.005579 21693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:08.040205 21693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:08.043351 21693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:08.043556 21693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:08.043902 21693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:08.044415 21722 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260620 11:41:08.046109 21722 ts_tablet_manager.cc:1852] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260620 11:41:08.046413 21722 ts_tablet_manager.cc:1905] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:08.049014 21722 ts_tablet_manager.cc:1918] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:08.049134 21722 log.cc:1196] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/d4f204bc0e124a23a70a992e4d44b7f6
I20260620 11:41:08.049412 21722 ts_tablet_manager.cc:1939] T d4f204bc0e124a23a70a992e4d44b7f6 P 2cad626b94154223aff38c15075f65e2: Deleting consensus metadata
I20260620 11:41:08.049643 21693 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260620 11:41:08.049721 21693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260620 11:41:08.049769 21693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:08.049815 21693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:08.056370 21693 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:41987
I20260620 11:41:08.056424 21829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:41987 every 8 connection(s)
I20260620 11:41:08.056785 21693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:08.057018 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21693
I20260620 11:41:08.057344 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:46761
--local_ip_for_outbound_sockets=127.5.122.131
--tserver_master_addrs=127.5.122.190:35853
--webserver_port=34055
--webserver_interface=127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 11:41:08 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955667, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:08.068837 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.058976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:49205 (local address 127.5.122.190:35853)
0620 11:41:08.059116 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.059118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.059127 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:41:08.059157 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.059159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.059174 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.059260 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:08.060013 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.060560 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.061912 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.062246 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.062348 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.065008 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:08.065012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:08.065013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:08.065075 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:08.066987 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.067710 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.067711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.067712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.067799 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.068070 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.068071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.068071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.068215 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:08.068297 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.068622 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.068733 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260620 11:41:08.069046 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.057684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:49205)
0620 11:41:08.057912 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:08.058013 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:08.058586 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:08.059361 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:08.059366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:08.059655 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:08.059872 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.059877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.060678 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.060680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.061651 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.061654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.062364 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.062365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.062518 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.063271 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:08.063282 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:08.064876 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:08.067162 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:08.067173 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:08.067176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:08.067602 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:08.067852 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:08.067853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:08.067853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:08.067992 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:08.068286 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:08.068294 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:08.068466 (+ 172us) client_negotiation.cc:769] Sending connection context
0620 11:41:08.068589 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 11:41:08.068778 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":104,"threads_started":1}
I20260620 11:41:08.069703 21830 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35853
I20260620 11:41:08.069828 21830 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:08.070109 21830 heartbeater.cc:511] Master 127.5.122.190:35853 requested a full tablet report, sending...
I20260620 11:41:08.070614 21503 ts_manager.cc:194] Registered new tserver with Master: 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
I20260620 11:41:08.071161 21503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:49205
W20260620 11:41:08.145474 21833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:08.145714 21833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:08.145742 21833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:08.146951 21833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:08.146996 21833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:08.147023 21833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:08.148543 21833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:46761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=34055
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:08.148792 21833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:08.148981 21833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:08.154431 21838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:08.154422 21839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:08.154466 21833 server_base.cc:1034] running on GCE node
W20260620 11:41:08.154417 21841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:08.154853 21833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:08.155156 21833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:08.156370 21833 hybrid_clock.cc:648] HybridClock initialized: now 1781955668156354 us; error 28 us; skew 500 ppm
Jun 20 11:41:08 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955668, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:08.159118 21833 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:41:08.159616 21833 webserver.cc:460] Webserver started at http://127.5.122.131:34055/ using document root <none> and password file <none>
I20260620 11:41:08.159866 21833 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:08.159950 21833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:08.161449 21833 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:08.162130 21848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:08.162318 21833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:08.162420 21833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "dbd879990fd9428e81e95344e52f6ee2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "626b0cca3c7b918801f018ce1c03de9d"
server_key_iv: "23f94f02be6209cd2b21a84105bc0afb"
server_key_version: "encryptionkey@0"
I20260620 11:41:08.162819 21833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:08.171247 21833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:08.173460 21833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:08.173645 21833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:08.173902 21833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:08.174317 21833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:08.174376 21833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:08.174422 21833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:08.174449 21833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:08.180593 21833 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:46761
I20260620 11:41:08.180639 21961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:46761 every 8 connection(s)
I20260620 11:41:08.181011 21833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:08.183164 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21833
Jun 20 11:41:08 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955668, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:08.190793 21966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.183624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.5.122.190:35853)
0620 11:41:08.183747 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.183749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.183840 (+ 91us) server_negotiation.cc:407] Connection header received
0620 11:41:08.183923 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.183924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.183937 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.183997 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:08.184590 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.185088 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.185898 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.186239 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.186311 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.187623 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:08.187625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:08.187625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:08.187633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:08.189335 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.189743 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.189744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.189744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.189812 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.190104 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.190105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.190105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.190219 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:08.190351 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.190601 (+ 250us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.190716 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260620 11:41:08.192593 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.181784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:36329 (local address 127.5.122.190:35853)
0620 11:41:08.181840 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.181843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.182040 (+ 197us) server_negotiation.cc:407] Connection header received
0620 11:41:08.182628 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.182629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.182643 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.182690 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:08.183854 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.184343 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.185415 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.185611 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.185692 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.188864 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:08.188866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:08.188866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:08.188874 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:08.190940 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.191378 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.191378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.191405 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.191486 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.191827 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.191828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.191829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.191981 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:08.192041 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.192398 (+ 357us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.192495 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:41:08.192749 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.181716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:36329)
0620 11:41:08.181875 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:08.181920 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:08.182501 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:08.182769 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:08.182773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:08.183447 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:08.183743 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.183752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.184453 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.184454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.185303 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.185305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.185727 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.185728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.185858 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.186554 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:08.186567 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:08.188743 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:08.191033 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:08.191042 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:08.191044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:08.191300 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:08.191557 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:08.191558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:08.191558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:08.191750 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:08.192049 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:08.192054 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:08.192248 (+ 194us) client_negotiation.cc:769] Sending connection context
0620 11:41:08.192342 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 11:41:08.192532 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260620 11:41:08.193346 21962 heartbeater.cc:347] Connected to a master server at 127.5.122.190:35853
I20260620 11:41:08.193444 21962 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:08.193724 21962 heartbeater.cc:511] Master 127.5.122.190:35853 requested a full tablet report, sending...
I20260620 11:41:08.194223 21503 ts_manager.cc:194] Registered new tserver with Master: dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761)
I20260620 11:41:08.194612 21503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:36329
I20260620 11:41:08.203044 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:41:08.206092 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.203528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47456 (local address 127.5.122.190:35853)
0620 11:41:08.203635 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.203637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.203721 (+ 84us) server_negotiation.cc:407] Connection header received
0620 11:41:08.203781 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.203781 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.203821 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.203896 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:08.204113 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.204611 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.205418 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.205600 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.205651 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.205787 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:08.205836 (+ 49us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 11:41:08.206026 (+ 190us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47456: token expired
Metrics: {"server-negotiator.queue_time_us":78}
W20260620 11:41:08.206176 21695 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47456: token expired
W20260620 11:41:08.206314 21432 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.122.190:35853)
I20260620 11:41:08.212509 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.206580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47466 (local address 127.5.122.190:35853)
0620 11:41:08.206652 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.206654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.206662 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:41:08.206681 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.206681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.206692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.206762 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:08.207301 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.207783 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.208493 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.208695 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.208796 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.210044 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:08.210046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:08.210046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:08.210054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:08.211306 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.211690 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.211691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.211691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.211757 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:08.211964 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:08.211964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:08.211965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:08.212088 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:08.212209 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.212315 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.212401 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 11:41:08.213768 21503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:41:08.215488 21503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:41:08.218012 21625 tablet_service.cc:1467] Processing CreateTablet for tablet 599df8a120c44745be9f09e34f15b363 (DEFAULT_TABLE table=test-table [id=831464a103f748299cd2941caba85f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:08.218273 21625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599df8a120c44745be9f09e34f15b363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:08.219960 21583 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap starting.
I20260620 11:41:08.220314 21583 tablet_bootstrap.cc:654] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:08.221158 21583 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: No bootstrap required, opened a new log
I20260620 11:41:08.221206 21583 ts_tablet_manager.cc:1397] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:08.221360 21583 raft_consensus.cc:348] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.221454 21583 raft_consensus.cc:374] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:08.221513 21583 raft_consensus.cc:729] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5518978f45465bb8ca9249ee7f1f96, State: Initialized, Role: FOLLOWER
I20260620 11:41:08.221557 21583 consensus_queue.cc:260] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.221662 21583 ts_tablet_manager.cc:1428] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:08.222239 21968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.217520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41987 (local address 127.0.0.1:59894)
0620 11:41:08.217723 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:08.217735 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:08.217792 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:08.218380 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:08.218381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:08.218384 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:08.218464 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.218466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.219742 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.219743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.220755 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.220758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.221865 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.221866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.221942 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.222021 (+ 79us) client_negotiation.cc:769] Sending connection context
0620 11:41:08.222079 (+ 58us) client_negotiation.cc:240] Negotiation successful
0620 11:41:08.222136 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":21,"thread_start_us":118,"threads_started":1}
I20260620 11:41:08.222240 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.217888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.5.122.130:41987)
0620 11:41:08.218067 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.218072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.218083 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:41:08.218149 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.218151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.218168 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.218254 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:08.218566 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.219649 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.220851 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.221765 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.221843 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.221904 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.222106 (+ 202us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.222149 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260620 11:41:08.222906 21764 tablet_service.cc:1467] Processing CreateTablet for tablet 599df8a120c44745be9f09e34f15b363 (DEFAULT_TABLE table=test-table [id=831464a103f748299cd2941caba85f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:08.223264 21764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599df8a120c44745be9f09e34f15b363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:08.223840 21698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.217520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:46761 (local address 127.0.0.1:50326)
0620 11:41:08.217578 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:08.217592 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:08.217632 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:08.220003 (+ 2371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:08.220005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:08.220008 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:08.220051 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.220052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.221435 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.221437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.222552 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.222555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.223608 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.223609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.223698 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.223703 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:08.223749 (+ 46us) client_negotiation.cc:240] Negotiation successful
0620 11:41:08.223788 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260620 11:41:08.223903 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.217652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.5.122.131:46761)
0620 11:41:08.217854 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.217860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.217872 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:41:08.217914 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.217916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.217933 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.218047 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:08.220168 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.221337 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.222646 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.223372 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.223589 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.223671 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.223764 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.223811 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260620 11:41:08.224722 21896 tablet_service.cc:1467] Processing CreateTablet for tablet 599df8a120c44745be9f09e34f15b363 (DEFAULT_TABLE table=test-table [id=831464a103f748299cd2941caba85f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:08.225208 21896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 599df8a120c44745be9f09e34f15b363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:08.225677 21722 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Bootstrap starting.
I20260620 11:41:08.226634 21722 tablet_bootstrap.cc:654] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:08.226995 21722 log.cc:824] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:08.227811 21722 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: No bootstrap required, opened a new log
I20260620 11:41:08.227877 21722 ts_tablet_manager.cc:1397] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:41:08.229228 21973 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Bootstrap starting.
I20260620 11:41:08.229949 21973 tablet_bootstrap.cc:654] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:08.229914 21722 raft_consensus.cc:348] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.230168 21722 raft_consensus.cc:374] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:08.230238 21722 raft_consensus.cc:729] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cad626b94154223aff38c15075f65e2, State: Initialized, Role: FOLLOWER
I20260620 11:41:08.230379 21722 consensus_queue.cc:260] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.230727 21830 heartbeater.cc:503] Master 127.5.122.190:35853 was elected leader, sending a full tablet report...
I20260620 11:41:08.230773 21973 log.cc:824] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:08.230715 21722 ts_tablet_manager.cc:1428] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:08.231771 21973 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: No bootstrap required, opened a new log
I20260620 11:41:08.231873 21973 ts_tablet_manager.cc:1397] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:41:08.233460 21973 raft_consensus.cc:348] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.233639 21973 raft_consensus.cc:374] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:08.233690 21973 raft_consensus.cc:729] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbd879990fd9428e81e95344e52f6ee2, State: Initialized, Role: FOLLOWER
I20260620 11:41:08.233819 21973 consensus_queue.cc:260] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.234131 21973 ts_tablet_manager.cc:1428] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 11:41:08.234155 21962 heartbeater.cc:503] Master 127.5.122.190:35853 was elected leader, sending a full tablet report...
W20260620 11:41:08.307685 21831 tablet.cc:2369] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:08.360987 21974 raft_consensus.cc:482] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:08.361168 21974 raft_consensus.cc:504] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.361625 21974 leader_election.cc:288] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761), 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371)
I20260620 11:41:08.364779 21969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.362031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:56697 (local address 127.5.122.131:46761)
0620 11:41:08.362082 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.362085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.362144 (+ 59us) server_negotiation.cc:407] Connection header received
0620 11:41:08.362211 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.362211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.362219 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.362262 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:08.362451 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.362964 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.363985 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.364404 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.364471 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.364479 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.364642 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.364689 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 11:41:08.364799 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.361875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:46761 (local address 127.5.122.130:56697)
0620 11:41:08.362078 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:08.362089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:08.362157 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:08.362290 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:08.362290 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:08.362293 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:08.362380 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.362382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.363056 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.363057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.363885 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.363888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.364489 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.364490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.364566 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.364571 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:08.364626 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 11:41:08.364686 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":1,"thread_start_us":125,"threads_started":1}
I20260620 11:41:08.364876 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.361912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:50697 (local address 127.5.122.129:46371)
0620 11:41:08.362005 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.362007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.362101 (+ 94us) server_negotiation.cc:407] Connection header received
0620 11:41:08.362209 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.362210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.362239 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.362315 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:08.362499 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.362971 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.364072 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.364501 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.364586 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.364682 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.364759 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.364801 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 11:41:08.365144 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.361856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46371 (local address 127.5.122.130:50697)
0620 11:41:08.361927 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:08.361939 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:08.362095 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:08.362331 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:08.362332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:08.362333 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:08.362398 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.362400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.363064 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.363065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.363878 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:08.363881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.364590 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.364591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:08.364683 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.364686 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:08.364750 (+ 64us) client_negotiation.cc:240] Negotiation successful
0620 11:41:08.364801 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":49}
I20260620 11:41:08.365201 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599df8a120c44745be9f09e34f15b363" candidate_uuid: "2cad626b94154223aff38c15075f65e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd879990fd9428e81e95344e52f6ee2" is_pre_election: true
I20260620 11:41:08.365401 21916 raft_consensus.cc:2454] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cad626b94154223aff38c15075f65e2 in term 0.
I20260620 11:41:08.365483 21645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599df8a120c44745be9f09e34f15b363" candidate_uuid: "2cad626b94154223aff38c15075f65e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5518978f45465bb8ca9249ee7f1f96" is_pre_election: true
I20260620 11:41:08.365658 21645 raft_consensus.cc:2454] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cad626b94154223aff38c15075f65e2 in term 0.
I20260620 11:41:08.365693 21718 leader_election.cc:302] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cad626b94154223aff38c15075f65e2, dbd879990fd9428e81e95344e52f6ee2; no voters:
I20260620 11:41:08.365818 21974 raft_consensus.cc:2781] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:41:08.365900 21974 raft_consensus.cc:482] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:41:08.365931 21974 raft_consensus.cc:3037] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:08.366839 21974 raft_consensus.cc:504] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.367009 21974 leader_election.cc:288] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [CANDIDATE]: Term 1 election: Requested vote from peers dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761), 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371)
I20260620 11:41:08.367175 21645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599df8a120c44745be9f09e34f15b363" candidate_uuid: "2cad626b94154223aff38c15075f65e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e5518978f45465bb8ca9249ee7f1f96"
I20260620 11:41:08.367209 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "599df8a120c44745be9f09e34f15b363" candidate_uuid: "2cad626b94154223aff38c15075f65e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dbd879990fd9428e81e95344e52f6ee2"
I20260620 11:41:08.367288 21645 raft_consensus.cc:3037] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:08.367280 21916 raft_consensus.cc:3037] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:08.368049 21916 raft_consensus.cc:2454] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cad626b94154223aff38c15075f65e2 in term 1.
I20260620 11:41:08.368111 21645 raft_consensus.cc:2454] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cad626b94154223aff38c15075f65e2 in term 1.
I20260620 11:41:08.368261 21718 leader_election.cc:302] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cad626b94154223aff38c15075f65e2, dbd879990fd9428e81e95344e52f6ee2; no voters:
I20260620 11:41:08.368366 21974 raft_consensus.cc:2781] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:08.368544 21974 raft_consensus.cc:686] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 1 LEADER]: Becoming Leader. State: Replica: 2cad626b94154223aff38c15075f65e2, State: Running, Role: LEADER
I20260620 11:41:08.368645 21974 consensus_queue.cc:237] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:08.369520 21503 catalog_manager.cc:5495] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cad626b94154223aff38c15075f65e2 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "2cad626b94154223aff38c15075f65e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } health_report { overall_health: UNKNOWN } } }
W20260620 11:41:08.390645 21692 tablet.cc:2369] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:08.411434 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.408923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59904 (local address 127.5.122.130:41987)
0620 11:41:08.408991 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:08.408992 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:08.409001 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:41:08.409142 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:08.409143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:08.409164 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:08.409232 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:08.409371 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.409934 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.410715 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:08.410901 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:08.410952 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:08.411112 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:08.411234 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:41:08.411299 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:08.411332 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 11:41:08.411351 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:41:08.413604 21916 raft_consensus.cc:1261] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 1 FOLLOWER]: Refusing update from remote peer 2cad626b94154223aff38c15075f65e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:08.413604 21645 raft_consensus.cc:1261] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Refusing update from remote peer 2cad626b94154223aff38c15075f65e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:08.414077 21974 consensus_queue.cc:1035] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:08.414235 21978 consensus_queue.cc:1035] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:08.416380 21981 mvcc.cc:205] Tried to move back new op lower bound from 7298890417818624000 to 7298890417638535168. Current Snapshot: MvccSnapshot[applied={T|T < 7298890417818624000}]
I20260620 11:41:08.416749 21982 mvcc.cc:205] Tried to move back new op lower bound from 7298890417818624000 to 7298890417638535168. Current Snapshot: MvccSnapshot[applied={T|T < 7298890417818624000}]
I20260620 11:41:08.419237 21502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:08.419356 21502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:08.420166 21502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Sending DeleteTablet for 3 replicas of tablet 599df8a120c44745be9f09e34f15b363
I20260620 11:41:08.420553 21625 tablet_service.cc:1514] Processing DeleteTablet for tablet 599df8a120c44745be9f09e34f15b363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:08 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:35752
I20260620 11:41:08.420528 21896 tablet_service.cc:1514] Processing DeleteTablet for tablet 599df8a120c44745be9f09e34f15b363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:08 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:50326
I20260620 11:41:08.420553 21764 tablet_service.cc:1514] Processing DeleteTablet for tablet 599df8a120c44745be9f09e34f15b363 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:08 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:59894
I20260620 11:41:08.420643 21700 tablet_replica.cc:330] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: stopping tablet replica
I20260620 11:41:08.420696 21700 raft_consensus.cc:2229] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:08.420747 21700 raft_consensus.cc:2258] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:08.420866 21991 tablet_replica.cc:330] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: stopping tablet replica
I20260620 11:41:08.420961 21991 raft_consensus.cc:2229] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:08.421111 21991 raft_consensus.cc:2258] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:08.421188 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 21541
I20260620 11:41:08.422173 21992 tablet_replica.cc:330] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: stopping tablet replica
I20260620 11:41:08.422312 21992 raft_consensus.cc:2229] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:41:08.422430 21992 raft_consensus.cc:2258] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:08.422912 21992 ts_tablet_manager.cc:1905] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:08.423416 21991 ts_tablet_manager.cc:1905] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:08.425171 21992 ts_tablet_manager.cc:1918] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:08.425259 21991 ts_tablet_manager.cc:1918] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:08.425348 21991 log.cc:1196] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/599df8a120c44745be9f09e34f15b363
I20260620 11:41:08.425354 21992 log.cc:1196] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/599df8a120c44745be9f09e34f15b363
I20260620 11:41:08.425674 21991 ts_tablet_manager.cc:1939] T 599df8a120c44745be9f09e34f15b363 P dbd879990fd9428e81e95344e52f6ee2: Deleting consensus metadata
I20260620 11:41:08.425787 21992 ts_tablet_manager.cc:1939] T 599df8a120c44745be9f09e34f15b363 P 2cad626b94154223aff38c15075f65e2: Deleting consensus metadata
I20260620 11:41:08.426308 21490 catalog_manager.cc:4841] TS 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987): tablet 599df8a120c44745be9f09e34f15b363 (table test-table [id=831464a103f748299cd2941caba85f48]) successfully deleted
I20260620 11:41:08.426340 21489 catalog_manager.cc:4841] TS dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761): tablet 599df8a120c44745be9f09e34f15b363 (table test-table [id=831464a103f748299cd2941caba85f48]) successfully deleted
W20260620 11:41:08.428283 21490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:46371 (error 108)
I20260620 11:41:08.428735 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 21693
I20260620 11:41:08.428858 21698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:08.428707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46371 (local address 127.0.0.1:35764)
0620 11:41:08.428772 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:08.428815 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 11:41:08.434122 21490 catalog_manager.cc:4584] TS 1e5518978f45465bb8ca9249ee7f1f96 (127.5.122.129:46371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 599df8a120c44745be9f09e34f15b363: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46371: connect: Connection refused (error 111)
I20260620 11:41:08.434962 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 21833
I20260620 11:41:08.440366 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 21470
I20260620 11:41:28.446643 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35853
--webserver_interface=127.5.122.190
--webserver_port=45883
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:35853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:28.533267 21994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:28.533510 21994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:28.533545 21994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:28.534885 21994 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 11:41:28.534931 21994 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 11:41:28.534950 21994 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 11:41:28.534962 21994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:28.534976 21994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:28.534989 21994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:28.535051 21994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:28.535069 21994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:28.536687 21994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:35853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:35853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=45883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:28.536993 21994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:28.537271 21994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:28.543006 22002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:28.543097 21994 server_base.cc:1034] running on GCE node
W20260620 11:41:28.543059 22000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:28.543061 21999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:28.543685 21994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:28.543987 21994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:28.545150 21994 hybrid_clock.cc:648] HybridClock initialized: now 1781955688545134 us; error 28 us; skew 500 ppm
Jun 20 11:41:28 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:28.548079 21994 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:41:28.548606 21994 webserver.cc:460] Webserver started at http://127.5.122.190:45883/ using document root <none> and password file <none>
I20260620 11:41:28.548866 21994 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:28.548949 21994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:28.550423 21994 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:28.551168 22009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:28.551374 21994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:28.551484 21994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "21fe7010f3d74440813517381035cd63"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "e4a465061f28c1795991d4c3f56313a9"
server_key_iv: "b4143d4a9546e142e5498d3b6517c5a8"
server_key_version: "encryptionkey@0"
I20260620 11:41:28.551860 21994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:28.569909 21994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:28.572791 21994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:28.572988 21994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:28.577776 21994 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:35853
I20260620 11:41:28.577818 22061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:35853 every 8 connection(s)
I20260620 11:41:28.578327 21994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:28.579962 22062 sys_catalog.cc:263] Verifying existing consensus state
I20260620 11:41:28.580622 22062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Bootstrap starting.
I20260620 11:41:28.582569 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 21994
I20260620 11:41:28.582913 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:46371
--local_ip_for_outbound_sockets=127.5.122.129
--tserver_master_addrs=127.5.122.190:35853
--webserver_port=43997
--webserver_interface=127.5.122.129
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:41:28.590466 22062 log.cc:824] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:28.592913 22062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:41:28.593309 22062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Bootstrap complete.
I20260620 11:41:28.596330 22062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:28.596586 22062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21fe7010f3d74440813517381035cd63, State: Initialized, Role: FOLLOWER
I20260620 11:41:28.596774 22062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:28.596858 22062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:28.596912 22062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:28.596962 22062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 2 FOLLOWER]: Advancing to term 3
I20260620 11:41:28.598107 22062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:28.598205 22062 leader_election.cc:302] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21fe7010f3d74440813517381035cd63; no voters:
I20260620 11:41:28.598425 22062 leader_election.cc:288] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 11:41:28.598467 22066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 11:41:28.598598 22066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [term 3 LEADER]: Becoming Leader. State: Replica: 21fe7010f3d74440813517381035cd63, State: Running, Role: LEADER
I20260620 11:41:28.598693 22066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } }
I20260620 11:41:28.598742 22062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:28.598994 22067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "21fe7010f3d74440813517381035cd63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } } }
I20260620 11:41:28.599089 22067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:28.598986 22066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21fe7010f3d74440813517381035cd63. Latest consensus state: current_term: 3 leader_uuid: "21fe7010f3d74440813517381035cd63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21fe7010f3d74440813517381035cd63" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 35853 } } }
I20260620 11:41:28.599261 22066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:28.599774 22071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:28.600718 22071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:28.600824 22071 catalog_manager.cc:1259] Loaded cluster ID: 8bdb81490e8045e78ae8b1ed1cb87b46
I20260620 11:41:28.600848 22071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:28.602134 22071 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:28.602331 22071 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Loaded TSK: 0
I20260620 11:41:28.602377 22071 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 21fe7010f3d74440813517381035cd63: Loaded TSK: 1
F20260620 11:41:28.602628 22071 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f14a89679ad google::LogMessage::Fail() at ??:0
@ 0x7f14a896a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f14a89674de google::LogMessage::Flush() at ??:0
@ 0x7f14a8968a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f14ab0d4104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f14ab0d1d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f14b08c11cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f14b08c148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f14b08c1594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f14b0887c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f14b0891ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f14aa8350d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f14aa82a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f14ab68e6db start_thread at ??:0
@ 0x7f14a746561f clone at ??:0
W20260620 11:41:28.686124 22064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:28.686369 22064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:28.686410 22064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:28.687692 22064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:28.687774 22064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:28.687822 22064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:28.689531 22064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:46371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=43997
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:28.689831 22064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:28.690153 22064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:28.695364 22106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:28.695380 22104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:28.695391 22103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:28.696005 22064 server_base.cc:1034] running on GCE node
I20260620 11:41:28.696244 22064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:28.696525 22064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:28.697698 22064 hybrid_clock.cc:648] HybridClock initialized: now 1781955688697678 us; error 40 us; skew 500 ppm
Jun 20 11:41:28 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:28.700549 22064 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:41:28.701157 22064 webserver.cc:460] Webserver started at http://127.5.122.129:43997/ using document root <none> and password file <none>
I20260620 11:41:28.701399 22064 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:28.701444 22064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:28.702939 22064 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:28.703685 22113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:28.703879 22064 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:28.703984 22064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "1e5518978f45465bb8ca9249ee7f1f96"
format_stamp: "Formatted at 2026-06-20 11:40:46 on dist-test-slave-szdr"
server_key: "b33cc4edebb9a2763937bdf438b54b4e"
server_key_iv: "907a52017e9de550ee54c5d1c54ff5bb"
server_key_version: "encryptionkey@0"
I20260620 11:41:28.704367 22064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:28.719933 22064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:28.722591 22064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:28.722721 22064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:28.723081 22064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:28.723647 22120 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260620 11:41:28.725257 22064 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 11:41:28.725322 22064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260620 11:41:28.725350 22064 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 11:41:28.726203 22064 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 11:41:28.726253 22064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:28.726305 22120 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap starting.
I20260620 11:41:28.734586 22064 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:46371
I20260620 11:41:28.734876 22227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:46371 every 8 connection(s)
I20260620 11:41:28.735247 22064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:41:28.738521 22120 log.cc:824] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:28.739023 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 22064
I20260620 11:41:28.739323 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:41987
--local_ip_for_outbound_sockets=127.5.122.130
--tserver_master_addrs=127.5.122.190:35853
--webserver_port=44209
--webserver_interface=127.5.122.130
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:41:28.742184 22120 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 11:41:28.742517 22120 tablet_bootstrap.cc:492] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Bootstrap complete.
I20260620 11:41:28.743824 22120 ts_tablet_manager.cc:1397] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260620 11:41:28.745466 22120 raft_consensus.cc:348] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:28.745855 22120 raft_consensus.cc:729] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5518978f45465bb8ca9249ee7f1f96, State: Initialized, Role: FOLLOWER
I20260620 11:41:28.746068 22120 consensus_queue.cc:260] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:28.746397 22120 ts_tablet_manager.cc:1428] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
W20260620 11:41:28.822372 22230 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:41:28.736065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:48073)
0620 11:41:28.736334 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:28.736393 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:28.737288 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:28.822156 (+ 84868us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":132,"threads_started":1}
W20260620 11:41:28.822726 22228 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:35853 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.190:35853: Client connection negotiation failed: client connection to 127.5.122.190:35853: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 11:41:28.823161 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:28.822996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:40641)
0620 11:41:28.823072 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:28.823131 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 11:41:28.823546 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:28.823437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:54591)
0620 11:41:28.823487 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:28.823520 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 11:41:28.823684 22228 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 11:41:28.833104 22232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:28.833359 22232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:28.833388 22232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:28.834802 22232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:28.834867 22232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:28.834899 22232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:28.836524 22232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:41987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=44209
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:28.836822 22232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:28.837091 22232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:28.842541 22239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:28.842572 22238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:28.842547 22241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:28.842854 22232 server_base.cc:1034] running on GCE node
I20260620 11:41:28.843143 22232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:28.843433 22232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:28.844609 22232 hybrid_clock.cc:648] HybridClock initialized: now 1781955688844601 us; error 48 us; skew 500 ppm
Jun 20 11:41:28 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:28.847479 22232 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:41:28.848062 22232 webserver.cc:460] Webserver started at http://127.5.122.130:44209/ using document root <none> and password file <none>
I20260620 11:41:28.848322 22232 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:28.848397 22232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:28.849852 22232 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260620 11:41:28.850529 22248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:28.850749 22232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:41:28.850863 22232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "2cad626b94154223aff38c15075f65e2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "73e24b3c860d5348907e046f295944d3"
server_key_iv: "2bb0046a2e33324fbecf33d913f8e5e7"
server_key_version: "encryptionkey@0"
I20260620 11:41:28.851248 22232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:28.882827 22232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:28.884922 22232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:28.885115 22232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:28.885380 22232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:28.885710 22232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:28.885766 22232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:28.885805 22232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:28.885852 22232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:28.892160 22232 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:41987
I20260620 11:41:28.892211 22361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:41987 every 8 connection(s)
I20260620 11:41:28.892611 22232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:28.893747 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:28.893329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:53437)
0620 11:41:28.893509 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:28.893590 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
W20260620 11:41:28.894253 22362 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:35853 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.190:35853: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
I20260620 11:41:28.894655 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:28.894492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:47411)
0620 11:41:28.894593 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:28.894629 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260620 11:41:28.895084 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:28.894979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:50305)
0620 11:41:28.895033 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:28.895061 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260620 11:41:28.895218 22362 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 11:41:28.895478 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 22232
I20260620 11:41:28.895743 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:46761
--local_ip_for_outbound_sockets=127.5.122.131
--tserver_master_addrs=127.5.122.190:35853
--webserver_port=34055
--webserver_interface=127.5.122.131
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:46199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:28.981063 22365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:28.981307 22365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:28.981336 22365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:28.982657 22365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:28.982708 22365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:28.982738 22365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:28.984303 22365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:46199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:46761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=34055
--tserver_master_addrs=127.5.122.190:35853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:28.984611 22365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:28.984835 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:28.990283 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:28.990365 22371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:28.990499 22365 server_base.cc:1034] running on GCE node
W20260620 11:41:28.990283 22370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:28.990792 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:28.991080 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:28.992261 22365 hybrid_clock.cc:648] HybridClock initialized: now 1781955688992255 us; error 36 us; skew 500 ppm
Jun 20 11:41:28 dist-test-slave-szdr krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:28.995123 22365 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:41:28.995646 22365 webserver.cc:460] Webserver started at http://127.5.122.131:34055/ using document root <none> and password file <none>
I20260620 11:41:28.995869 22365 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:28.995915 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:28.997349 22365 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:28.997946 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:28.998190 22365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:28.998286 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "dbd879990fd9428e81e95344e52f6ee2"
format_stamp: "Formatted at 2026-06-20 11:40:47 on dist-test-slave-szdr"
server_key: "626b0cca3c7b918801f018ce1c03de9d"
server_key_iv: "23f94f02be6209cd2b21a84105bc0afb"
server_key_version: "encryptionkey@0"
I20260620 11:41:28.998662 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:29.022994 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:29.025362 22365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:29.025532 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:29.025815 22365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:29.026208 22365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:29.026245 22365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:29.026268 22365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:29.026281 22365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:29.032469 22365 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:46761
I20260620 11:41:29.032524 22493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:46761 every 8 connection(s)
I20260620 11:41:29.032912 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:29.034224 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:29.033730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:53929)
0620 11:41:29.033905 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:29.034043 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
W20260620 11:41:29.034740 22494 heartbeater.cc:644] Failed to heartbeat to 127.5.122.190:35853 (0 consecutive failures): Network error: Failed to ping master at 127.5.122.190:35853: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
I20260620 11:41:29.035086 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:29.034985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:36563)
0620 11:41:29.035036 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:29.035066 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 11:41:29.035422 22496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:29.035324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:34065)
0620 11:41:29.035380 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:29.035404 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260620 11:41:29.035703 22494 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 11:41:29.041927 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 22365
I20260620 11:41:29.824440 22498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:29.824155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:41397)
0620 11:41:29.824325 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:29.824376 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260620 11:41:29.895864 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:29.895588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:36469)
0620 11:41:29.895761 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:29.895796 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260620 11:41:30.036332 22500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:30.036096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:55153)
0620 11:41:30.036222 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:30.036264 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260620 11:41:30.399156 22501 raft_consensus.cc:482] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:30.399307 22501 raft_consensus.cc:504] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:30.399669 22501 leader_election.cc:288] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761), 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
Jun 20 11:41:30 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.130@KRBTEST.COM
Jun 20 11:41:30 dist-test-slave-szdr krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955688, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.131@KRBTEST.COM
I20260620 11:41:30.411460 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:30.400057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:55439 (local address 127.5.122.131:46761)
0620 11:41:30.400252 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:30.400260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:30.400286 (+ 26us) server_negotiation.cc:407] Connection header received
0620 11:41:30.400739 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:30.400742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:30.400788 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:30.400957 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:30.401852 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.402917 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.403986 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.404405 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.404541 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:30.407779 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:30.407784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:30.407784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:30.407797 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:30.409790 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:30.410222 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:30.410225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:30.410226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:30.410326 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:30.410540 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:30.410541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:30.410541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:30.410797 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:30.410889 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:30.411180 (+ 291us) server_negotiation.cc:299] Negotiation successful
0620 11:41:30.411325 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260620 11:41:30.411445 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:30.399864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:46761 (local address 127.5.122.129:55439)
0620 11:41:30.400052 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:30.400067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:30.400165 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:30.400991 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:30.400994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:30.401500 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:30.401713 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:30.401719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.403034 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.403035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:30.403887 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:30.403889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.404504 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.404505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:30.404620 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:30.405346 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:30.405353 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:30.407663 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:30.409875 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:30.409876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:30.409876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:30.410125 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:30.410398 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:30.410399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:30.410400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:30.410463 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:30.410880 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:30.410887 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:30.411012 (+ 125us) client_negotiation.cc:769] Sending connection context
0620 11:41:30.411125 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 11:41:30.411338 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":21,"thread_start_us":77,"threads_started":1}
I20260620 11:41:30.411473 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:30.400017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:54853 (local address 127.5.122.130:41987)
0620 11:41:30.400206 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:30.400221 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:30.400246 (+ 25us) server_negotiation.cc:407] Connection header received
0620 11:41:30.400739 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:30.400742 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:30.400788 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:30.400955 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:30.401851 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.402917 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.403994 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.404406 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.404541 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:30.407376 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:30.407383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:30.407384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:30.407398 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:30.409545 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:30.410019 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:30.410022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:30.410022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:30.410133 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:30.410397 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:30.410398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:30.410399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:30.410803 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:30.410893 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:30.411191 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 11:41:30.411353 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260620 11:41:30.411559 22503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:30.399920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:41987 (local address 127.5.122.129:54853)
0620 11:41:30.400057 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:30.400069 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:30.400133 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:30.400991 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:30.400994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:30.401501 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:30.401713 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:30.401719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.403026 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.403028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:30.403886 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:30.403888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:30.404502 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:30.404503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:30.404621 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:30.405343 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:30.405352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:30.407239 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:30.409638 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:30.409644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:30.409646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:30.409867 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:30.410211 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:30.410212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:30.410212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:30.410324 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:30.410886 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:30.410892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:30.411011 (+ 119us) client_negotiation.cc:769] Sending connection context
0620 11:41:30.411126 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 11:41:30.411421 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":183,"thread_start_us":77,"threads_started":1}
W20260620 11:41:30.412572 22117 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
W20260620 11:41:30.413079 22116 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
I20260620 11:41:30.413152 22116 leader_election.cc:302] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e5518978f45465bb8ca9249ee7f1f96; no voters: 2cad626b94154223aff38c15075f65e2, dbd879990fd9428e81e95344e52f6ee2
I20260620 11:41:30.413273 22501 raft_consensus.cc:2726] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:41:30.825349 22503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:30.825115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:50587)
0620 11:41:30.825240 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:30.825294 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260620 11:41:30.896734 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:30.896438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:59805)
0620 11:41:30.896619 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:30.896677 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260620 11:41:31.037151 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:31.036902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:57203)
0620 11:41:31.037055 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:31.037101 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":64,"threads_started":1}
I20260620 11:41:31.826243 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:31.825916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:33035)
0620 11:41:31.826136 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:31.826187 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":125,"threads_started":1}
I20260620 11:41:31.897799 22509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:31.897450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:48949)
0620 11:41:31.897680 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:31.897741 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":287,"threads_started":1}
I20260620 11:41:32.038115 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:32.037806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:38315)
0620 11:41:32.038010 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:32.038054 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260620 11:41:32.091324 22511 raft_consensus.cc:482] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:32.091485 22511 raft_consensus.cc:504] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:32.091691 22511 leader_election.cc:288] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761), 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
W20260620 11:41:32.092139 22116 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
W20260620 11:41:32.092226 22117 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
I20260620 11:41:32.092344 22117 leader_election.cc:302] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e5518978f45465bb8ca9249ee7f1f96; no voters: 2cad626b94154223aff38c15075f65e2, dbd879990fd9428e81e95344e52f6ee2
I20260620 11:41:32.092478 22511 raft_consensus.cc:2726] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:41:32.827083 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:32.826850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:58587)
0620 11:41:32.826995 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:32.827035 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":67,"threads_started":1}
I20260620 11:41:32.898819 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:32.898529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:57567)
0620 11:41:32.898714 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:32.898766 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260620 11:41:33.039001 22514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:33.038706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:33305)
0620 11:41:33.038897 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:33.038950 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
I20260620 11:41:33.827991 22515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:33.827711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:60539)
0620 11:41:33.827891 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:33.827941 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":72,"threads_started":1}
I20260620 11:41:33.899765 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:33.899488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:56345)
0620 11:41:33.899667 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:33.899715 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260620 11:41:34.039945 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:34.039633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:53417)
0620 11:41:34.039776 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:34.039892 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260620 11:41:34.828860 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:34.828586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:43029)
0620 11:41:34.828755 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:34.828804 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260620 11:41:34.900718 22519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:34.900465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:60935)
0620 11:41:34.900619 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:34.900669 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260620 11:41:35.040838 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:35.040574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:58319)
0620 11:41:35.040729 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:35.040782 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260620 11:41:35.249574 22521 raft_consensus.cc:482] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:35.249676 22521 raft_consensus.cc:504] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:35.249938 22521 leader_election.cc:288] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761), 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
W20260620 11:41:35.250411 22117 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
W20260620 11:41:35.250499 22116 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
I20260620 11:41:35.250593 22116 leader_election.cc:302] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e5518978f45465bb8ca9249ee7f1f96; no voters: 2cad626b94154223aff38c15075f65e2, dbd879990fd9428e81e95344e52f6ee2
I20260620 11:41:35.250701 22521 raft_consensus.cc:2726] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:41:35.829873 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:35.829499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:45827)
0620 11:41:35.829736 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:35.829785 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":132,"threads_started":1}
I20260620 11:41:35.901584 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:35.901305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:58467)
0620 11:41:35.901479 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:35.901532 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260620 11:41:36.041831 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:36.041508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:39913)
0620 11:41:36.041710 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:36.041765 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":128,"threads_started":1}
I20260620 11:41:36.830674 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:36.830424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:44129)
0620 11:41:36.830573 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:36.830606 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260620 11:41:36.902712 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:36.902338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:56761)
0620 11:41:36.902609 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:36.902660 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":122,"threads_started":1}
I20260620 11:41:37.042687 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:37.042442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:47073)
0620 11:41:37.042583 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:37.042635 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260620 11:41:37.195010 22528 raft_consensus.cc:482] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:37.195111 22528 raft_consensus.cc:504] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:37.195328 22528 leader_election.cc:288] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761), 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
W20260620 11:41:37.195741 22116 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
W20260620 11:41:37.195871 22117 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
I20260620 11:41:37.195940 22117 leader_election.cc:302] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e5518978f45465bb8ca9249ee7f1f96; no voters: 2cad626b94154223aff38c15075f65e2, dbd879990fd9428e81e95344e52f6ee2
I20260620 11:41:37.196012 22528 raft_consensus.cc:2726] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:41:37.831465 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:37.831208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:44733)
0620 11:41:37.831370 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:37.831417 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
I20260620 11:41:37.903589 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:37.903327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:60319)
0620 11:41:37.903486 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:37.903536 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260620 11:41:38.043570 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:38.043307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:51725)
0620 11:41:38.043461 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:38.043514 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260620 11:41:38.832480 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:38.832180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:46957)
0620 11:41:38.832373 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:38.832423 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":116,"threads_started":1}
I20260620 11:41:38.904376 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:38.904121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:56449)
0620 11:41:38.904276 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:38.904322 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260620 11:41:38.992179 22534 raft_consensus.cc:482] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:38.992290 22534 raft_consensus.cc:504] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dbd879990fd9428e81e95344e52f6ee2" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 46761 } } peers { permanent_uuid: "2cad626b94154223aff38c15075f65e2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 41987 } } peers { permanent_uuid: "1e5518978f45465bb8ca9249ee7f1f96" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46371 } }
I20260620 11:41:38.992631 22534 leader_election.cc:288] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761), 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987)
W20260620 11:41:38.993125 22117 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2cad626b94154223aff38c15075f65e2 (127.5.122.130:41987): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
W20260620 11:41:38.993248 22116 leader_election.cc:341] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dbd879990fd9428e81e95344e52f6ee2 (127.5.122.131:46761): Not found: Tablet not found: 599df8a120c44745be9f09e34f15b363
I20260620 11:41:38.993289 22116 leader_election.cc:302] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e5518978f45465bb8ca9249ee7f1f96; no voters: 2cad626b94154223aff38c15075f65e2, dbd879990fd9428e81e95344e52f6ee2
I20260620 11:41:38.993405 22534 raft_consensus.cc:2726] T 599df8a120c44745be9f09e34f15b363 P 1e5518978f45465bb8ca9249ee7f1f96 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 11:41:39.044543 22535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:39.044249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:51089)
0620 11:41:39.044439 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:39.044488 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
I20260620 11:41:39.833451 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:39.833170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:42153)
0620 11:41:39.833341 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:39.833393 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260620 11:41:39.905150 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:39.904916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:50431)
0620 11:41:39.905055 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:39.905100 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260620 11:41:40.045432 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:40.045203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:42245)
0620 11:41:40.045341 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:40.045383 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260620 11:41:40.834299 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:40.834001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:53467)
0620 11:41:40.834202 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:40.834247 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
I20260620 11:41:40.906051 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:40.905726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:52717)
0620 11:41:40.905879 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:40.905948 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260620 11:41:41.046216 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:41.045946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:34045)
0620 11:41:41.046126 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:41.046165 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":118,"threads_started":1}
I20260620 11:41:41.835227 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:41.834912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:37605)
0620 11:41:41.835090 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:41.835139 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":103,"threads_started":1}
I20260620 11:41:41.906939 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:41.906602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:58107)
0620 11:41:41.906820 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:41.906877 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":117,"threads_started":1}
I20260620 11:41:42.047159 22544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:42.046813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:45149)
0620 11:41:42.047036 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:42.047108 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":87,"threads_started":1}
I20260620 11:41:42.835950 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:42.835676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:32817)
0620 11:41:42.835859 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:42.835902 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260620 11:41:42.908413 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:42.908092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:42139)
0620 11:41:42.908261 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:42.908318 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260620 11:41:43.048044 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:43.047741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:48591)
0620 11:41:43.047934 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:43.047986 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260620 11:41:43.836925 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:43.836670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.129:50495)
0620 11:41:43.836823 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:43.836872 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260620 11:41:43.909345 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:43.909068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.130:46295)
0620 11:41:43.909238 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:43.909294 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260620 11:41:44.049050 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:44.048796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:35853 (local address 127.5.122.131:43563)
0620 11:41:44.048953 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:44.049001 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.190:35853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260620 11:41:44.051337 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 22064
I20260620 11:41:44.056723 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 22232
I20260620 11:41:44.061465 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 22365
2026-06-20T11:41:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58606 ms)
[----------] 30 tests from SecurityITest (237920 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:44 dist-test-slave-szdr krb5kdc[22553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:44 dist-test-slave-szdr krb5kdc[22553](info): set up 2 sockets
Jun 20 11:41:44 dist-test-slave-szdr krb5kdc[22553](info): commencing operation
krb5kdc: starting...
W20260620 11:41:45.125859 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:41:45.258370 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:42391
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:42391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:45.345115 22566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:45.345357 22566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:45.345382 22566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:45.346601 22566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:45.346647 22566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:45.346662 22566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:45.346674 22566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:45.346688 22566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:45.348165 22566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:42391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:42391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:45.348372 22566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:45.348562 22566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:45.353799 22574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:45.353799 22571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:45.353835 22572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:45.354329 22566 server_base.cc:1034] running on GCE node
I20260620 11:41:45.354589 22566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:45.354914 22566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:45.356092 22566 hybrid_clock.cc:648] HybridClock initialized: now 1781955705356042 us; error 77 us; skew 500 ppm
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:45.358975 22566 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:41:45.359509 22566 webserver.cc:460] Webserver started at http://127.5.122.190:45405/ using document root <none> and password file <none>
I20260620 11:41:45.359752 22566 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:45.359822 22566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:45.359988 22566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:45.361130 22566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "5bb657845f50436eb211e52e7f3958b0"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "87882ec2b8dd39f98378e8021d7eae89"
server_key_iv: "14b479a414c09628c036c45c352342b7"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.361577 22566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "5bb657845f50436eb211e52e7f3958b0"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "87882ec2b8dd39f98378e8021d7eae89"
server_key_iv: "14b479a414c09628c036c45c352342b7"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.363204 22566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:45.364131 22581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.364372 22566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:45.364519 22566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "5bb657845f50436eb211e52e7f3958b0"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "87882ec2b8dd39f98378e8021d7eae89"
server_key_iv: "14b479a414c09628c036c45c352342b7"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.364605 22566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:45.380141 22566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:45.383054 22566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:45.383263 22566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:45.387604 22566 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:42391
I20260620 11:41:45.387638 22633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:42391 every 8 connection(s)
I20260620 11:41:45.388022 22566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:45.388906 22634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:45.391561 22634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0: Bootstrap starting.
I20260620 11:41:45.392210 22634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:45.392530 22634 log.cc:824] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:45.393365 22634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0: No bootstrap required, opened a new log
I20260620 11:41:45.394054 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 22566
I20260620 11:41:45.394169 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:45.394371 5610 external_mini_cluster.cc:1428] Setting key ada204e892f713d3a952c228375484a3
I20260620 11:41:45.395756 22634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb657845f50436eb211e52e7f3958b0" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42391 } }
I20260620 11:41:45.395953 22634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:45.395998 22634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bb657845f50436eb211e52e7f3958b0, State: Initialized, Role: FOLLOWER
I20260620 11:41:45.396181 22634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb657845f50436eb211e52e7f3958b0" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42391 } }
I20260620 11:41:45.396282 22634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:45.396317 22634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:45.396369 22634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:45.397468 22634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb657845f50436eb211e52e7f3958b0" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42391 } }
I20260620 11:41:45.397574 22634 leader_election.cc:302] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bb657845f50436eb211e52e7f3958b0; no voters:
I20260620 11:41:45.397779 22634 leader_election.cc:288] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:45.397838 22639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:45.398120 22639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [term 1 LEADER]: Becoming Leader. State: Replica: 5bb657845f50436eb211e52e7f3958b0, State: Running, Role: LEADER
I20260620 11:41:45.398124 22634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:45.398241 22639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb657845f50436eb211e52e7f3958b0" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42391 } }
I20260620 11:41:45.398816 22639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bb657845f50436eb211e52e7f3958b0. Latest consensus state: current_term: 1 leader_uuid: "5bb657845f50436eb211e52e7f3958b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb657845f50436eb211e52e7f3958b0" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42391 } } }
I20260620 11:41:45.399021 22639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:45.398960 22640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bb657845f50436eb211e52e7f3958b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bb657845f50436eb211e52e7f3958b0" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42391 } } }
I20260620 11:41:45.399206 22640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:45.399307 22648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:45.400092 22648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:45.401997 22648 catalog_manager.cc:1347] Generated new cluster ID: 1c64d7916e4841c6ba4ac5e85920fea5
I20260620 11:41:45.402053 22648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:45.406665 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.395028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.5.122.190:42391)
0620 11:41:45.395242 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.395250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.395278 (+ 28us) server_negotiation.cc:407] Connection header received
0620 11:41:45.395736 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.395743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.395796 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.395922 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:45.396747 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.397815 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.399058 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.399800 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.399894 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.402927 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:45.402931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:45.402931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:45.402943 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:45.404732 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.405132 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.405136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.405136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.405261 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.405494 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.405495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.405495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.405770 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:45.406003 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.406104 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.406267 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260620 11:41:45.430774 22648 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:45.431394 22648 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:45.435515 22648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0: Generated new TSK 0
I20260620 11:41:45.435716 22648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:41:45.471096 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42391
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:45.576937 22660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:45.577170 22660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:45.577198 22660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:45.578516 22660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:45.578570 22660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:45.578600 22660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:45.580168 22660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:45.580403 22660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:45.580606 22660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:45.586146 22666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:45.586301 22668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:45.586170 22665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:45.586510 22660 server_base.cc:1034] running on GCE node
I20260620 11:41:45.586738 22660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:45.587034 22660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:45.588181 22660 hybrid_clock.cc:648] HybridClock initialized: now 1781955705588150 us; error 45 us; skew 500 ppm
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:45.591005 22660 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:41:45.591553 22660 webserver.cc:460] Webserver started at http://127.5.122.129:37559/ using document root <none> and password file <none>
I20260620 11:41:45.591816 22660 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:45.591869 22660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:45.592025 22660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:45.593171 22660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "1c57c8db1a504b119b7e1645ea9c46e6"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "495c454f42fab0459ec205b18a620240"
server_key_iv: "c2d90861114a48289b57246ff15563b3"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.593538 22660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "1c57c8db1a504b119b7e1645ea9c46e6"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "495c454f42fab0459ec205b18a620240"
server_key_iv: "c2d90861114a48289b57246ff15563b3"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.595120 22660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:45.596108 22675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.596321 22660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:45.596429 22660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "1c57c8db1a504b119b7e1645ea9c46e6"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "495c454f42fab0459ec205b18a620240"
server_key_iv: "c2d90861114a48289b57246ff15563b3"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.596511 22660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:45.625403 22660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:45.627707 22660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:45.627969 22660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:45.628244 22660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:45.628602 22660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:45.628638 22660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.628660 22660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:45.628705 22660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.635210 22660 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:38891
I20260620 11:41:45.635251 22788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:38891 every 8 connection(s)
I20260620 11:41:45.635614 22660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:41:45.637717 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 22660
I20260620 11:41:45.637826 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:41:45.638099 5610 external_mini_cluster.cc:1428] Setting key 63766f6568d09a6fb4e82f9ba048286a
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:45.646873 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.636518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:46179 (local address 127.5.122.190:42391)
0620 11:41:45.636589 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.636591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.636812 (+ 221us) server_negotiation.cc:407] Connection header received
0620 11:41:45.637456 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.637458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.637475 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.637530 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:45.638899 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.639405 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.640478 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.640708 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.640801 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.643263 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:45.643266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:45.643266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:45.643279 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:45.645140 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.645634 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.645636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.645637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.645717 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.646031 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.646032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.646033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.646198 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:45.646280 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.646648 (+ 368us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.646762 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:41:45.646997 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.636476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:42391 (local address 127.5.122.129:46179)
0620 11:41:45.636652 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:45.636704 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:45.637311 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:45.637651 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:45.637655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:45.638472 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:45.638772 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.638779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.639510 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.639511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.640368 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.640371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.640813 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.640814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.640969 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.641588 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:45.641597 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:45.643125 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:45.645261 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:45.645270 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:45.645272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:45.645541 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:45.645792 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:45.645792 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:45.645793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:45.645917 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:45.646304 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:45.646312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:45.646480 (+ 168us) client_negotiation.cc:769] Sending connection context
0620 11:41:45.646589 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 11:41:45.646744 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
I20260620 11:41:45.647573 22789 heartbeater.cc:347] Connected to a master server at 127.5.122.190:42391
I20260620 11:41:45.647706 22789 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:45.647944 22789 heartbeater.cc:511] Master 127.5.122.190:42391 requested a full tablet report, sending...
I20260620 11:41:45.648612 22598 ts_manager.cc:194] Registered new tserver with Master: 1c57c8db1a504b119b7e1645ea9c46e6 (127.5.122.129:38891)
I20260620 11:41:45.649295 22598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:46179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:41:45.666851 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42391
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:45.752430 22794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:45.752657 22794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:45.752684 22794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:45.754002 22794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:45.754051 22794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:45.754081 22794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:45.755563 22794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:45.755790 22794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:45.755993 22794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:45.761601 22802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:45.761749 22794 server_base.cc:1034] running on GCE node
W20260620 11:41:45.761649 22800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:45.761691 22799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:45.762092 22794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:45.762372 22794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:45.763542 22794 hybrid_clock.cc:648] HybridClock initialized: now 1781955705763516 us; error 41 us; skew 500 ppm
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:45.766491 22794 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:41:45.767087 22794 webserver.cc:460] Webserver started at http://127.5.122.130:36671/ using document root <none> and password file <none>
I20260620 11:41:45.767392 22794 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:45.767443 22794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:45.767545 22794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:45.768765 22794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "cb74af7cb14b48bcb8480cc96c55419f"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "95dae3657d50b08207f8fde3166b11c1"
server_key_iv: "5d90082a71a61ebd944183b63ae097fb"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.769088 22794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "cb74af7cb14b48bcb8480cc96c55419f"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "95dae3657d50b08207f8fde3166b11c1"
server_key_iv: "5d90082a71a61ebd944183b63ae097fb"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.770767 22794 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260620 11:41:45.771567 22809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.771768 22794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:45.771869 22794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "cb74af7cb14b48bcb8480cc96c55419f"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "95dae3657d50b08207f8fde3166b11c1"
server_key_iv: "5d90082a71a61ebd944183b63ae097fb"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.771931 22794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:45.782860 22794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:45.785010 22794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:45.785131 22794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:45.785326 22794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:45.785658 22794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:45.785694 22794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.785718 22794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:45.785764 22794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.792069 22794 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:36627
I20260620 11:41:45.792110 22922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:36627 every 8 connection(s)
I20260620 11:41:45.792470 22794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:45.792644 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 22794
I20260620 11:41:45.792760 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:41:45.792980 5610 external_mini_cluster.cc:1428] Setting key bff0c94f577a9aa82dd2d7c93c413beb
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
I20260620 11:41:45.803488 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.793417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:36835 (local address 127.5.122.190:42391)
0620 11:41:45.793491 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.793494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.793745 (+ 251us) server_negotiation.cc:407] Connection header received
0620 11:41:45.794392 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.794393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.794405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.794453 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:45.795469 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.795987 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.797022 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.797240 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.797323 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.799953 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:45.799955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:45.799955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:45.799964 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:45.801743 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.802297 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.802298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.802298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.802366 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.802659 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.802660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.802660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.802838 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:45.802908 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.803286 (+ 378us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.803389 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260620 11:41:45.803742 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.793378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:42391 (local address 127.5.122.130:36835)
0620 11:41:45.793587 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:45.793640 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:45.794276 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:45.794529 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:45.794533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:45.795146 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:45.795358 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.795363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.796089 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.796090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.796926 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.796929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.797336 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.797337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.797474 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.798094 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:45.798104 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:45.799841 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:45.801830 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:45.801839 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:45.801842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:45.802168 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:45.802452 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:45.802453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:45.802453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:45.802581 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:45.802896 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:45.802904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:45.803065 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 11:41:45.803200 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 11:41:45.803393 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260620 11:41:45.804505 22923 heartbeater.cc:347] Connected to a master server at 127.5.122.190:42391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
I20260620 11:41:45.804638 22923 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:45.804839 22923 heartbeater.cc:511] Master 127.5.122.190:42391 requested a full tablet report, sending...
I20260620 11:41:45.805400 22598 ts_manager.cc:194] Registered new tserver with Master: cb74af7cb14b48bcb8480cc96c55419f (127.5.122.130:36627)
I20260620 11:41:45.805862 22598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:36835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:41:45.821923 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42391
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:45.910475 22928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:45.910758 22928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:45.910791 22928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:45.912245 22928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:45.912307 22928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:45.912340 22928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:45.914144 22928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:45.914405 22928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:45.914634 22928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:45.920558 22933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:45.920558 22936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:45.920637 22928 server_base.cc:1034] running on GCE node
W20260620 11:41:45.920859 22934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:45.921169 22928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:45.921530 22928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:45.922708 22928 hybrid_clock.cc:648] HybridClock initialized: now 1781955705922676 us; error 54 us; skew 500 ppm
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:45.925796 22928 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:41:45.926329 22928 webserver.cc:460] Webserver started at http://127.5.122.131:35041/ using document root <none> and password file <none>
I20260620 11:41:45.926596 22928 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:45.926685 22928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:45.926844 22928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:45.927980 22928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "f955e43ca2bf4b48a8b72ef7474ee902"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "97cd94b31a17678f2e250dd2af52fc51"
server_key_iv: "14777b3e8cc8eb817a8b1985f244b914"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.928375 22928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "f955e43ca2bf4b48a8b72ef7474ee902"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "97cd94b31a17678f2e250dd2af52fc51"
server_key_iv: "14777b3e8cc8eb817a8b1985f244b914"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.929844 22928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:41:45.930835 22943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.931051 22928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:45.931176 22928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "f955e43ca2bf4b48a8b72ef7474ee902"
format_stamp: "Formatted at 2026-06-20 11:41:45 on dist-test-slave-szdr"
server_key: "97cd94b31a17678f2e250dd2af52fc51"
server_key_iv: "14777b3e8cc8eb817a8b1985f244b914"
server_key_version: "encryptionkey@0"
I20260620 11:41:45.931267 22928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:45.945779 22928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:45.948594 22928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:45.948796 22928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:45.949070 22928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:45.949414 22928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:45.949470 22928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.949518 22928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:45.949553 22928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:45.956009 22928 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:36137
I20260620 11:41:45.956058 23056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:36137 every 8 connection(s)
I20260620 11:41:45.956429 22928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:45.957741 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 22928
I20260620 11:41:45.957864 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:41:45.958067 5610 external_mini_cluster.cc:1428] Setting key bde7be99303d4da5040f27f88578d67b
Jun 20 11:41:45 dist-test-slave-szdr krb5kdc[22553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955705, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:45.967012 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.957178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:54107 (local address 127.5.122.190:42391)
0620 11:41:45.957270 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.957273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.957566 (+ 293us) server_negotiation.cc:407] Connection header received
0620 11:41:45.958266 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.958267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.958281 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.958339 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:45.959406 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.959927 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.961012 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.961211 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.961294 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.963723 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:45.963724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:45.963725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:45.963733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:45.965321 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.965832 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.965833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.965834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.965906 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.966225 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.966226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.966226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.966380 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:45.966462 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.966803 (+ 341us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.966899 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 11:41:45.967235 23059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.957147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:42391 (local address 127.5.122.131:54107)
0620 11:41:45.957386 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:45.957456 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:45.958163 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:45.958429 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:45.958434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:45.959076 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:45.959286 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.959291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.960035 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.960036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.960909 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.960912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.961302 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.961304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.961462 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.962040 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:45.962048 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:45.963607 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:45.965440 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:45.965449 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:45.965451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:45.965738 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:45.966009 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:45.966010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:45.966011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:45.966141 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:45.966463 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:45.966471 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:45.966631 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 11:41:45.966763 (+ 132us) client_negotiation.cc:240] Negotiation successful
0620 11:41:45.966953 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":123,"threads_started":1}
I20260620 11:41:45.967836 23057 heartbeater.cc:347] Connected to a master server at 127.5.122.190:42391
I20260620 11:41:45.967968 23057 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:45.968222 23057 heartbeater.cc:511] Master 127.5.122.190:42391 requested a full tablet report, sending...
I20260620 11:41:45.968652 22598 ts_manager.cc:194] Registered new tserver with Master: f955e43ca2bf4b48a8b72ef7474ee902 (127.5.122.131:36137)
I20260620 11:41:45.969118 22598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:54107
I20260620 11:41:45.970041 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:41:45.977644 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.971401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.5.122.190:42391)
0620 11:41:45.971470 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.971472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.971634 (+ 162us) server_negotiation.cc:407] Connection header received
0620 11:41:45.971668 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.971669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.971679 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.971729 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:45.972290 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.972776 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.973487 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.973674 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.973731 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.975079 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:45.975080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:45.975081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:45.975087 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:45.976578 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.976900 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.976900 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.976901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.976953 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:45.977099 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:45.977100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:45.977100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:45.977235 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:45.977306 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.977483 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.977549 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 11:41:45.979308 22598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:41:45.980986 22598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:41:45.987463 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.983500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:36627 (local address 127.0.0.1:59034)
0620 11:41:45.983614 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:45.983655 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:45.983724 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:45.984094 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:45.984097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:45.984099 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:45.984178 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.984181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.985136 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.985138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.986335 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.986337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.987141 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.987142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.987231 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.987246 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 11:41:45.987324 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 11:41:45.987377 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":66,"threads_started":1}
I20260620 11:41:45.987484 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.983700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.5.122.130:36627)
0620 11:41:45.983893 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.983898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.983909 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:41:45.983955 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.983957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.983975 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.984082 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:45.984239 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.985028 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.986425 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.987050 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.987129 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.987191 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.987354 (+ 163us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.987395 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260620 11:41:45.987962 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.983507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:36137 (local address 127.0.0.1:37520)
0620 11:41:45.984170 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:45.984180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:45.984278 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:45.984516 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:45.984517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:45.984519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:45.984566 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.984567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.985540 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.985542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.986804 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.986807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.987732 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.987733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.987792 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.987796 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:45.987839 (+ 43us) client_negotiation.cc:240] Negotiation successful
0620 11:41:45.987881 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":97,"threads_started":1}
I20260620 11:41:45.988217 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.983909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37520 (local address 127.5.122.131:36137)
0620 11:41:45.984120 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.984124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.984248 (+ 124us) server_negotiation.cc:407] Connection header received
0620 11:41:45.984358 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.984360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.984377 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.984446 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:45.984650 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.985418 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.986871 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.987558 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.987633 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.987687 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.988085 (+ 398us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.988133 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260620 11:41:45.988725 22857 tablet_service.cc:1467] Processing CreateTablet for tablet 83cbef4508344e1d92b575ff99059803 (DEFAULT_TABLE table=test-table [id=986eb144fc204e02998f37e4af8176a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:45.989018 23070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.983730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:38891 (local address 127.0.0.1:50904)
0620 11:41:45.985185 (+ 1455us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:45.985196 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:45.985247 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:45.985447 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:45.985448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:45.985450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:45.985487 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.985489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.986790 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.986791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.987807 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:45.987810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.988779 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.988780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:45.988853 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.988858 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:45.988910 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:41:45.988953 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1433,"thread_start_us":77,"threads_started":1}
I20260620 11:41:45.988943 22991 tablet_service.cc:1467] Processing CreateTablet for tablet 83cbef4508344e1d92b575ff99059803 (DEFAULT_TABLE table=test-table [id=986eb144fc204e02998f37e4af8176a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:45.989142 22857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83cbef4508344e1d92b575ff99059803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:45.989264 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:45.983867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50904 (local address 127.5.122.129:38891)
0620 11:41:45.984067 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:45.984073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:45.985296 (+ 1223us) server_negotiation.cc:407] Connection header received
0620 11:41:45.985344 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:45.985347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:45.985362 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:45.985436 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:45.985548 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.986681 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.987900 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:45.988615 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:45.988688 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:45.988741 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:45.989128 (+ 387us) server_negotiation.cc:299] Negotiation successful
0620 11:41:45.989182 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260620 11:41:45.989310 22991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83cbef4508344e1d92b575ff99059803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:45.990815 22723 tablet_service.cc:1467] Processing CreateTablet for tablet 83cbef4508344e1d92b575ff99059803 (DEFAULT_TABLE table=test-table [id=986eb144fc204e02998f37e4af8176a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:45.991178 22723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83cbef4508344e1d92b575ff99059803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:45.992241 23074 tablet_bootstrap.cc:492] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Bootstrap starting.
I20260620 11:41:45.992858 23075 tablet_bootstrap.cc:492] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Bootstrap starting.
I20260620 11:41:45.993041 23074 tablet_bootstrap.cc:654] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:45.993621 23075 tablet_bootstrap.cc:654] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:45.993660 23074 log.cc:824] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:45.994243 23075 log.cc:824] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:45.994436 23077 tablet_bootstrap.cc:492] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6: Bootstrap starting.
I20260620 11:41:45.994580 23074 tablet_bootstrap.cc:492] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: No bootstrap required, opened a new log
I20260620 11:41:45.994668 23074 ts_tablet_manager.cc:1397] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:41:45.994997 23075 tablet_bootstrap.cc:492] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: No bootstrap required, opened a new log
I20260620 11:41:45.995090 23075 ts_tablet_manager.cc:1397] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 11:41:45.995262 23077 tablet_bootstrap.cc:654] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:45.995775 23077 log.cc:824] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:45.996497 23077 tablet_bootstrap.cc:492] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6: No bootstrap required, opened a new log
I20260620 11:41:45.996587 23077 ts_tablet_manager.cc:1397] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:41:45.996785 23074 raft_consensus.cc:348] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:45.997001 23074 raft_consensus.cc:374] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:45.997052 23074 raft_consensus.cc:729] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb74af7cb14b48bcb8480cc96c55419f, State: Initialized, Role: FOLLOWER
I20260620 11:41:45.997094 23075 raft_consensus.cc:348] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:45.997260 23075 raft_consensus.cc:374] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:45.997223 23074 consensus_queue.cc:260] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:45.997303 23075 raft_consensus.cc:729] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f955e43ca2bf4b48a8b72ef7474ee902, State: Initialized, Role: FOLLOWER
I20260620 11:41:45.997429 23075 consensus_queue.cc:260] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:45.997519 23074 ts_tablet_manager.cc:1428] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:45.997670 22923 heartbeater.cc:503] Master 127.5.122.190:42391 was elected leader, sending a full tablet report...
I20260620 11:41:45.997725 23075 ts_tablet_manager.cc:1428] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 11:41:45.997797 23057 heartbeater.cc:503] Master 127.5.122.190:42391 was elected leader, sending a full tablet report...
I20260620 11:41:45.999159 23077 raft_consensus.cc:348] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:45.999336 23077 raft_consensus.cc:374] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:45.999362 23077 raft_consensus.cc:729] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c57c8db1a504b119b7e1645ea9c46e6, State: Initialized, Role: FOLLOWER
I20260620 11:41:45.999476 23077 consensus_queue.cc:260] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:45.999818 23077 ts_tablet_manager.cc:1428] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:45.999846 22789 heartbeater.cc:503] Master 127.5.122.190:42391 was elected leader, sending a full tablet report...
I20260620 11:41:46.023346 23081 raft_consensus.cc:482] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:46.023511 23081 raft_consensus.cc:504] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:46.023916 23081 leader_election.cc:288] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb74af7cb14b48bcb8480cc96c55419f (127.5.122.130:36627), 1c57c8db1a504b119b7e1645ea9c46e6 (127.5.122.129:38891)
I20260620 11:41:46.028285 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.024268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:53735 (local address 127.5.122.130:36627)
0620 11:41:46.024315 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.024317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.024464 (+ 147us) server_negotiation.cc:407] Connection header received
0620 11:41:46.024512 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.024513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.024525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.024565 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:46.024721 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.025441 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.027211 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.027866 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.027960 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.027971 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.028153 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.028206 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:41:46.028292 23059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.024134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:38891 (local address 127.5.122.131:42165)
0620 11:41:46.024210 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.024223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.024268 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.024454 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.024455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.024457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:46.024540 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.024542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.025551 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.025552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.026997 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.027002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.027967 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.027968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.028058 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.028060 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.028120 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.028180 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":96}
I20260620 11:41:46.028337 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.024238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:42165 (local address 127.5.122.129:38891)
0620 11:41:46.024294 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.024296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.024310 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:41:46.024344 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.024345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.024363 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.024451 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:46.024621 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.025450 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.027261 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.027867 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.027961 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.027971 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.028203 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.028256 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 11:41:46.028303 23083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.024145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:36627 (local address 127.5.122.131:53735)
0620 11:41:46.024369 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.024377 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.024451 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.024588 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.024589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.024590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:46.024656 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.024657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.025551 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.025552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.026997 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.027002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.027968 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.027968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.028054 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.028060 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.028120 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.028180 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":70,"thread_start_us":120,"threads_started":1}
I20260620 11:41:46.028777 22743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cbef4508344e1d92b575ff99059803" candidate_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" is_pre_election: true
I20260620 11:41:46.028759 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cbef4508344e1d92b575ff99059803" candidate_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb74af7cb14b48bcb8480cc96c55419f" is_pre_election: true
I20260620 11:41:46.028996 22743 raft_consensus.cc:2454] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f955e43ca2bf4b48a8b72ef7474ee902 in term 0.
I20260620 11:41:46.029002 22877 raft_consensus.cc:2454] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f955e43ca2bf4b48a8b72ef7474ee902 in term 0.
I20260620 11:41:46.029299 22944 leader_election.cc:302] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c57c8db1a504b119b7e1645ea9c46e6, f955e43ca2bf4b48a8b72ef7474ee902; no voters:
I20260620 11:41:46.029505 23081 raft_consensus.cc:2781] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:41:46.029554 23081 raft_consensus.cc:482] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:41:46.029570 23081 raft_consensus.cc:3037] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:46.030403 23081 raft_consensus.cc:504] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:46.030638 23081 leader_election.cc:288] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [CANDIDATE]: Term 1 election: Requested vote from peers cb74af7cb14b48bcb8480cc96c55419f (127.5.122.130:36627), 1c57c8db1a504b119b7e1645ea9c46e6 (127.5.122.129:38891)
I20260620 11:41:46.030767 22743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cbef4508344e1d92b575ff99059803" candidate_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c57c8db1a504b119b7e1645ea9c46e6"
I20260620 11:41:46.030854 22877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83cbef4508344e1d92b575ff99059803" candidate_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb74af7cb14b48bcb8480cc96c55419f"
I20260620 11:41:46.030886 22743 raft_consensus.cc:3037] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:46.030970 22877 raft_consensus.cc:3037] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:46.031817 22743 raft_consensus.cc:2454] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f955e43ca2bf4b48a8b72ef7474ee902 in term 1.
I20260620 11:41:46.031826 22877 raft_consensus.cc:2454] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f955e43ca2bf4b48a8b72ef7474ee902 in term 1.
I20260620 11:41:46.032011 22944 leader_election.cc:302] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c57c8db1a504b119b7e1645ea9c46e6, f955e43ca2bf4b48a8b72ef7474ee902; no voters:
I20260620 11:41:46.032125 23081 raft_consensus.cc:2781] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:46.032286 23081 raft_consensus.cc:686] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 1 LEADER]: Becoming Leader. State: Replica: f955e43ca2bf4b48a8b72ef7474ee902, State: Running, Role: LEADER
I20260620 11:41:46.032400 23081 consensus_queue.cc:237] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } }
I20260620 11:41:46.033427 22596 catalog_manager.cc:5495] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 reported cstate change: term changed from 0 to 1, leader changed from <none> to f955e43ca2bf4b48a8b72ef7474ee902 (127.5.122.131). New cstate: current_term: 1 leader_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f955e43ca2bf4b48a8b72ef7474ee902" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 } health_report { overall_health: UNKNOWN } } }
I20260620 11:41:46.040810 23073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.038532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37524 (local address 127.5.122.131:36137)
0620 11:41:46.038586 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.038588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.038596 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:41:46.038667 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.038668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.038686 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.038748 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:46.038862 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.039344 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.040094 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.040320 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.040379 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.040495 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:46.040606 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:41:46.040672 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.040686 (+ 14us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.040704 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 11:41:46.042963 22877 raft_consensus.cc:1261] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 1 FOLLOWER]: Refusing update from remote peer f955e43ca2bf4b48a8b72ef7474ee902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:46.043049 22743 raft_consensus.cc:1261] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 1 FOLLOWER]: Refusing update from remote peer f955e43ca2bf4b48a8b72ef7474ee902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260620 11:41:46.043124 22924 tablet.cc:2369] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:46.043483 23084 consensus_queue.cc:1035] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb74af7cb14b48bcb8480cc96c55419f" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:46.043627 23081 consensus_queue.cc:1035] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c57c8db1a504b119b7e1645ea9c46e6" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:46.046300 23087 mvcc.cc:205] Tried to move back new op lower bound from 7298890571948593152 to 7298890571909652480. Current Snapshot: MvccSnapshot[applied={T|T < 7298890571948593152}]
I20260620 11:41:46.047777 23089 mvcc.cc:205] Tried to move back new op lower bound from 7298890571948593152 to 7298890571909652480. Current Snapshot: MvccSnapshot[applied={T|T < 7298890571948593152}]
I20260620 11:41:46.049002 22597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:46.049095 22597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:46.049823 22597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bb657845f50436eb211e52e7f3958b0: Sending DeleteTablet for 3 replicas of tablet 83cbef4508344e1d92b575ff99059803
I20260620 11:41:46.050195 22857 tablet_service.cc:1514] Processing DeleteTablet for tablet 83cbef4508344e1d92b575ff99059803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:46 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:59034
I20260620 11:41:46.050246 22723 tablet_service.cc:1514] Processing DeleteTablet for tablet 83cbef4508344e1d92b575ff99059803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:46 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:50904
I20260620 11:41:46.050424 23096 tablet_replica.cc:330] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: stopping tablet replica
I20260620 11:41:46.050498 23097 tablet_replica.cc:330] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6: stopping tablet replica
I20260620 11:41:46.050490 22991 tablet_service.cc:1514] Processing DeleteTablet for tablet 83cbef4508344e1d92b575ff99059803 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:46 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:37520
I20260620 11:41:46.050540 23096 raft_consensus.cc:2229] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:46.050581 23097 raft_consensus.cc:2229] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:46.050652 23096 raft_consensus.cc:2258] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:46.050663 23097 raft_consensus.cc:2258] T 83cbef4508344e1d92b575ff99059803 P 1c57c8db1a504b119b7e1645ea9c46e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:46.050701 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 22660
I20260620 11:41:46.051609 23096 ts_tablet_manager.cc:1905] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:46.053644 23096 ts_tablet_manager.cc:1918] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:46.053702 23096 log.cc:1196] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/83cbef4508344e1d92b575ff99059803
I20260620 11:41:46.054015 23096 ts_tablet_manager.cc:1939] T 83cbef4508344e1d92b575ff99059803 P cb74af7cb14b48bcb8480cc96c55419f: Deleting consensus metadata
I20260620 11:41:46.054523 22583 catalog_manager.cc:4841] TS cb74af7cb14b48bcb8480cc96c55419f (127.5.122.130:36627): tablet 83cbef4508344e1d92b575ff99059803 (table test-table [id=986eb144fc204e02998f37e4af8176a7]) successfully deleted
I20260620 11:41:46.058135 23098 tablet_replica.cc:330] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: stopping tablet replica
I20260620 11:41:46.058283 23098 raft_consensus.cc:2229] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:41:46.058410 23098 raft_consensus.cc:2258] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:46.059108 23098 ts_tablet_manager.cc:1905] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 11:41:46.059432 22582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:38891 (error 108)
I20260620 11:41:46.060019 23070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.059861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:38891 (local address 127.0.0.1:50920)
0620 11:41:46.059924 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.059972 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:38891: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260620 11:41:46.060134 22582 catalog_manager.cc:4584] TS 1c57c8db1a504b119b7e1645ea9c46e6 (127.5.122.129:38891): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83cbef4508344e1d92b575ff99059803: Network error: Client connection negotiation failed: client connection to 127.5.122.129:38891: connect: Connection refused (error 111)
I20260620 11:41:46.060364 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 22794
I20260620 11:41:46.061159 23098 ts_tablet_manager.cc:1918] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:46.061239 23098 log.cc:1196] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/83cbef4508344e1d92b575ff99059803
I20260620 11:41:46.061532 23098 ts_tablet_manager.cc:1939] T 83cbef4508344e1d92b575ff99059803 P f955e43ca2bf4b48a8b72ef7474ee902: Deleting consensus metadata
I20260620 11:41:46.062018 22582 catalog_manager.cc:4841] TS f955e43ca2bf4b48a8b72ef7474ee902 (127.5.122.131:36137): tablet 83cbef4508344e1d92b575ff99059803 (table test-table [id=986eb144fc204e02998f37e4af8176a7]) successfully deleted
I20260620 11:41:46.068344 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 22928
I20260620 11:41:46.073599 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 22566
2026-06-20T11:41:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2012 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T11:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:46Z Disabled control of system clock
I20260620 11:41:46.102849 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:46005
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:35643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:46005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:46.191583 23103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:46.191848 23103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:46.191872 23103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:46.193065 23103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:46.193111 23103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:46.193125 23103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:46.193137 23103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:46.193150 23103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:46.194617 23103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:46005
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:46005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:46.194857 23103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:46.195137 23103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:46.200608 23109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:46.200590 23111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:46.200837 23103 server_base.cc:1034] running on GCE node
W20260620 11:41:46.200590 23108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:46.201511 23103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:46.201855 23103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:46.203006 23103 hybrid_clock.cc:648] HybridClock initialized: now 1781955706203000 us; error 54 us; skew 500 ppm
I20260620 11:41:46.204494 23103 webserver.cc:460] Webserver started at http://127.5.122.190:38575/ using document root <none> and password file <none>
I20260620 11:41:46.204805 23103 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:46.204859 23103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:46.205031 23103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:46.206254 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "b185e16b06624132b6091dedfffcf2ac"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "7994ebf43b9c24a054a18efd3e4851f3"
server_key_iv: "8b6a5c50f50969592b8d5fdc654efe00"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.206645 23103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "b185e16b06624132b6091dedfffcf2ac"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "7994ebf43b9c24a054a18efd3e4851f3"
server_key_iv: "8b6a5c50f50969592b8d5fdc654efe00"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.208074 23103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:46.208842 23117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.209131 23103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:46.209241 23103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "b185e16b06624132b6091dedfffcf2ac"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "7994ebf43b9c24a054a18efd3e4851f3"
server_key_iv: "8b6a5c50f50969592b8d5fdc654efe00"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.209322 23103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:46.219151 23103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:46.219517 23103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:46.219691 23103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:46.223753 23103 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:46005
I20260620 11:41:46.223801 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:46005 every 8 connection(s)
I20260620 11:41:46.224161 23103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:46.225178 23170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:46.227437 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 23103
I20260620 11:41:46.227564 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:46.227756 5610 external_mini_cluster.cc:1428] Setting key 53bec1de11b60e8a7e8ba4d714627bd9
I20260620 11:41:46.227941 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac: Bootstrap starting.
I20260620 11:41:46.228653 23170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:46.228976 23170 log.cc:824] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:46.229691 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac: No bootstrap required, opened a new log
I20260620 11:41:46.231637 23170 raft_consensus.cc:348] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185e16b06624132b6091dedfffcf2ac" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46005 } }
I20260620 11:41:46.231801 23170 raft_consensus.cc:374] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:46.231824 23170 raft_consensus.cc:729] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b185e16b06624132b6091dedfffcf2ac, State: Initialized, Role: FOLLOWER
I20260620 11:41:46.231971 23170 consensus_queue.cc:260] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185e16b06624132b6091dedfffcf2ac" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46005 } }
I20260620 11:41:46.232049 23170 raft_consensus.cc:388] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:46.232071 23170 raft_consensus.cc:482] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:46.232198 23170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:46.232815 23170 raft_consensus.cc:504] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185e16b06624132b6091dedfffcf2ac" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46005 } }
I20260620 11:41:46.232937 23170 leader_election.cc:302] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b185e16b06624132b6091dedfffcf2ac; no voters:
I20260620 11:41:46.233163 23170 leader_election.cc:288] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:46.233193 23175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:46.233309 23175 raft_consensus.cc:686] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [term 1 LEADER]: Becoming Leader. State: Replica: b185e16b06624132b6091dedfffcf2ac, State: Running, Role: LEADER
I20260620 11:41:46.233454 23175 consensus_queue.cc:237] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185e16b06624132b6091dedfffcf2ac" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46005 } }
I20260620 11:41:46.233459 23170 sys_catalog.cc:564] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:46.233919 23177 sys_catalog.cc:455] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b185e16b06624132b6091dedfffcf2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185e16b06624132b6091dedfffcf2ac" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46005 } } }
I20260620 11:41:46.233903 23176 sys_catalog.cc:455] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader b185e16b06624132b6091dedfffcf2ac. Latest consensus state: current_term: 1 leader_uuid: "b185e16b06624132b6091dedfffcf2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b185e16b06624132b6091dedfffcf2ac" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 46005 } } }
I20260620 11:41:46.234028 23177 sys_catalog.cc:458] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:46.234045 23176 sys_catalog.cc:458] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:46.233893 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.228364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.5.122.190:46005)
0620 11:41:46.228597 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.228604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.228630 (+ 26us) server_negotiation.cc:407] Connection header received
0620 11:41:46.229151 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.229159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.229226 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.229379 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:46.229589 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.230640 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.231490 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.232065 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.232200 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.232928 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:46.232933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:46.232933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:46.233158 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:46.233193 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:46.233223 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:46.233293 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.233555 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.233639 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260620 11:41:46.234267 23181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:46.234922 23181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:46.236548 23181 catalog_manager.cc:1347] Generated new cluster ID: 96ef332e31f14f30821ba140f227b913
I20260620 11:41:46.236609 23181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:46.259205 23181 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:46.259936 23181 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:46.265830 23181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac: Generated new TSK 0
I20260620 11:41:46.266170 23181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:46.269774 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46005
--builtin_ntp_servers=127.5.122.148:35643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:46.377506 23194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:46.377760 23194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:46.377785 23194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:46.379113 23194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:46.379173 23194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:46.379204 23194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:46.380686 23194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:46.380980 23194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:46.381333 23194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:46.387823 23199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:46.387925 23202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:46.388229 23194 server_base.cc:1034] running on GCE node
W20260620 11:41:46.387828 23200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:46.389407 23194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:46.389730 23194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:46.390923 23194 hybrid_clock.cc:648] HybridClock initialized: now 1781955706390911 us; error 30 us; skew 500 ppm
I20260620 11:41:46.392674 23194 webserver.cc:460] Webserver started at http://127.5.122.129:39021/ using document root <none> and password file <none>
I20260620 11:41:46.392971 23194 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:46.393024 23194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:46.393167 23194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:46.394451 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "cd8e53e4baf0451683ad2b023ac360c7"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "daad7475d64f5ef64de3251a4c8bc7cf"
server_key_iv: "ebc6f71f935b0a18cfaee2e377e5248d"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.394843 23194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "cd8e53e4baf0451683ad2b023ac360c7"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "daad7475d64f5ef64de3251a4c8bc7cf"
server_key_iv: "ebc6f71f935b0a18cfaee2e377e5248d"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.396366 23194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:46.397302 23208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.397534 23194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:46.397651 23194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "cd8e53e4baf0451683ad2b023ac360c7"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "daad7475d64f5ef64de3251a4c8bc7cf"
server_key_iv: "ebc6f71f935b0a18cfaee2e377e5248d"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.397763 23194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:46.413168 23194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:46.413537 23194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:46.413699 23194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:46.414026 23194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:46.414418 23194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:46.414474 23194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.414510 23194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:46.414538 23194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.420810 23194 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:46373
I20260620 11:41:46.420851 23321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:46373 every 8 connection(s)
I20260620 11:41:46.421221 23194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:41:46.426545 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 23194
I20260620 11:41:46.426685 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:41:46.426947 5610 external_mini_cluster.cc:1428] Setting key f0875e5ffc6574dc67c90f3066a1ede5
I20260620 11:41:46.427373 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.422064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:58351 (local address 127.5.122.190:46005)
0620 11:41:46.422124 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.422127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.422450 (+ 323us) server_negotiation.cc:407] Connection header received
0620 11:41:46.422963 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.422964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.422975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.423045 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:46.423839 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.424387 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.425470 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.425741 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.425801 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.426918 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:46.426920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:46.426920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:46.426928 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:46.426946 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:46.426951 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:46.427023 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.427253 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.427287 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260620 11:41:46.427637 23324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.422016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:46005 (local address 127.5.122.129:58351)
0620 11:41:46.422301 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.422340 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.422865 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.423129 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.423132 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.423141 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:46.423670 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.423674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.424476 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.424477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.425338 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.425340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.425830 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.425831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.426018 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.426731 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:46.426732 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:46.426778 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:46.426812 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:46.427069 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:46.427080 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:46.427084 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.427252 (+ 168us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.427378 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":164,"threads_started":1}
I20260620 11:41:46.428149 23322 heartbeater.cc:347] Connected to a master server at 127.5.122.190:46005
I20260620 11:41:46.428246 23322 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:46.428313 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46005
--builtin_ntp_servers=127.5.122.148:35643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 11:41:46.428490 23322 heartbeater.cc:511] Master 127.5.122.190:46005 requested a full tablet report, sending...
I20260620 11:41:46.429067 23134 ts_manager.cc:194] Registered new tserver with Master: cd8e53e4baf0451683ad2b023ac360c7 (127.5.122.129:46373)
I20260620 11:41:46.429952 23134 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.129:58351
W20260620 11:41:46.515165 23325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:46.515412 23325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:46.515439 23325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:46.516670 23325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:46.516716 23325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:46.516749 23325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:46.518239 23325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:46.518468 23325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:46.518762 23325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:46.524488 23333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:46.524518 23331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:46.524488 23330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:46.524896 23325 server_base.cc:1034] running on GCE node
I20260620 11:41:46.525152 23325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:46.525429 23325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:46.526597 23325 hybrid_clock.cc:648] HybridClock initialized: now 1781955706526585 us; error 44 us; skew 500 ppm
I20260620 11:41:46.528066 23325 webserver.cc:460] Webserver started at http://127.5.122.130:34957/ using document root <none> and password file <none>
I20260620 11:41:46.528347 23325 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:46.528422 23325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:46.528576 23325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:46.529788 23325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "e469aede980f4f31a521142c12c501a4"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "91f29c3de4b55608231d5e3da1654726"
server_key_iv: "9095e565e2fb52b827eef228ff47316c"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.530211 23325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "e469aede980f4f31a521142c12c501a4"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "91f29c3de4b55608231d5e3da1654726"
server_key_iv: "9095e565e2fb52b827eef228ff47316c"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.531672 23325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:46.532507 23339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.532760 23325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:46.532872 23325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "e469aede980f4f31a521142c12c501a4"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "91f29c3de4b55608231d5e3da1654726"
server_key_iv: "9095e565e2fb52b827eef228ff47316c"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.532971 23325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:46.545251 23325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:46.545639 23325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:46.545823 23325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:46.546195 23325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:46.546573 23325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:46.546630 23325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.546674 23325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:46.546703 23325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.553107 23325 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:33263
I20260620 11:41:46.553153 23452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:33263 every 8 connection(s)
I20260620 11:41:46.553544 23325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:46.554418 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 23325
I20260620 11:41:46.554522 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:41:46.554723 5610 external_mini_cluster.cc:1428] Setting key bbd8b617ce9f7c22093774178b4f6d0c
I20260620 11:41:46.555826 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46005
--builtin_ntp_servers=127.5.122.148:35643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 11:41:46.559933 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.554453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:53667 (local address 127.5.122.190:46005)
0620 11:41:46.554542 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.554545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.554877 (+ 332us) server_negotiation.cc:407] Connection header received
0620 11:41:46.555539 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.555540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.555553 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.555617 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:46.556454 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.556969 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.558092 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.558401 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.558476 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.559474 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:46.559476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:46.559477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:46.559484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:46.559504 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:46.559510 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:46.559578 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.559765 (+ 187us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.559804 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 11:41:46.560056 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.554456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:46005 (local address 127.5.122.130:53667)
0620 11:41:46.554693 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.554739 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.555424 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.555709 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.555714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.555735 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:46.556337 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.556343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.557084 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.557085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.557988 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.557992 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.558501 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.558502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.558660 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.559297 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:46.559297 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:46.559334 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:46.559362 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:46.559619 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:46.559627 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:46.559631 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.559738 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.559837 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260620 11:41:46.560622 23453 heartbeater.cc:347] Connected to a master server at 127.5.122.190:46005
I20260620 11:41:46.560760 23453 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:46.561003 23453 heartbeater.cc:511] Master 127.5.122.190:46005 requested a full tablet report, sending...
I20260620 11:41:46.561453 23134 ts_manager.cc:194] Registered new tserver with Master: e469aede980f4f31a521142c12c501a4 (127.5.122.130:33263)
I20260620 11:41:46.561870 23134 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.130:53667
W20260620 11:41:46.642722 23456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:46.642962 23456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:46.643016 23456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:46.644412 23456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:46.644465 23456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:46.644495 23456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:46.646044 23456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:35643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:46005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:46.646347 23456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:46.646711 23456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:46.652768 23462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:46.652724 23461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:46.652745 23464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:46.653038 23456 server_base.cc:1034] running on GCE node
I20260620 11:41:46.653291 23456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:46.653578 23456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:46.654945 23456 hybrid_clock.cc:648] HybridClock initialized: now 1781955706654928 us; error 28 us; skew 500 ppm
I20260620 11:41:46.656469 23456 webserver.cc:460] Webserver started at http://127.5.122.131:46659/ using document root <none> and password file <none>
I20260620 11:41:46.656785 23456 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:46.656862 23456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:46.657028 23456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:46.658305 23456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "afc471bf38094277873d1130f9851b9c"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "6a7362cd947b130fc10030147e69683f"
server_key_iv: "2f1198a83dbc0138de98566f7d73b540"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.658685 23456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "afc471bf38094277873d1130f9851b9c"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "6a7362cd947b130fc10030147e69683f"
server_key_iv: "2f1198a83dbc0138de98566f7d73b540"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.660049 23456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:46.660869 23470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.661049 23456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260620 11:41:46.661149 23456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "afc471bf38094277873d1130f9851b9c"
format_stamp: "Formatted at 2026-06-20 11:41:46 on dist-test-slave-szdr"
server_key: "6a7362cd947b130fc10030147e69683f"
server_key_iv: "2f1198a83dbc0138de98566f7d73b540"
server_key_version: "encryptionkey@0"
I20260620 11:41:46.661234 23456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:46.682346 23456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:46.682713 23456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:46.682883 23456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:46.683163 23456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:46.683537 23456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:46.683595 23456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.683638 23456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:46.683669 23456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:46.689801 23456 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:38039
I20260620 11:41:46.689893 23583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:38039 every 8 connection(s)
I20260620 11:41:46.690322 23456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:46.692201 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 23456
I20260620 11:41:46.692315 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:41:46.692505 5610 external_mini_cluster.cc:1428] Setting key 405948e7be513925eb2a1a3e54434215
I20260620 11:41:46.696583 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.691203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:41085 (local address 127.5.122.190:46005)
0620 11:41:46.691297 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.691299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.691523 (+ 224us) server_negotiation.cc:407] Connection header received
0620 11:41:46.692108 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.692109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.692121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.692193 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:46.693206 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.693718 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.694828 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.695049 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.695140 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.696233 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:46.696234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:46.696235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:46.696241 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:46.696259 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:46.696266 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:46.696337 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.696438 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.696471 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 11:41:46.696842 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.691165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:46005 (local address 127.5.122.131:41085)
0620 11:41:46.691357 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.691402 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.691920 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.692319 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.692324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.692337 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:46.693107 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.693115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.693820 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.693821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.694733 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.694735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.695158 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.695160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.695308 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.695983 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:46.695983 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:46.696022 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:46.696098 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:46.696339 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:46.696345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:46.696350 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.696450 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.696573 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260620 11:41:46.697415 23584 heartbeater.cc:347] Connected to a master server at 127.5.122.190:46005
I20260620 11:41:46.697564 23584 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:46.697779 23584 heartbeater.cc:511] Master 127.5.122.190:46005 requested a full tablet report, sending...
I20260620 11:41:46.698248 23134 ts_manager.cc:194] Registered new tserver with Master: afc471bf38094277873d1130f9851b9c (127.5.122.131:38039)
I20260620 11:41:46.698655 23134 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.131:41085
I20260620 11:41:46.704698 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:41:46.709515 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.706359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43386 (local address 127.5.122.190:46005)
0620 11:41:46.706424 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.706427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.706558 (+ 131us) server_negotiation.cc:407] Connection header received
0620 11:41:46.706645 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.706646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.706656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.706727 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:46.706914 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.707458 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.708265 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.708462 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.708542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.709235 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:46.709237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:46.709237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:46.709243 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:46.709259 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:46.709270 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:46.709325 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.709392 (+ 67us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.709418 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:41:46.711383 23134 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:41:46.713318 23134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:41:46.720548 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.716250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.5.122.129:46373)
0620 11:41:46.716782 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.716786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.716796 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:41:46.716833 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.716835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.716850 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.717089 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:46.717164 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.717863 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.719420 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.720048 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.720171 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.720290 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.720381 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.720429 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":108,"threads_started":1}
I20260620 11:41:46.720738 23594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.715916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46373 (local address 127.0.0.1:45248)
0620 11:41:46.716439 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.716452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.716514 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.716944 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.716944 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.716946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:46.717013 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.717014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.717981 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.717983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.719314 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.719317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.720172 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.720177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.720271 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.720286 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.720366 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.720664 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":45,"thread_start_us":62,"threads_started":1}
I20260620 11:41:46.720834 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.716234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.5.122.130:33263)
0620 11:41:46.716415 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.716421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.716433 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:41:46.716480 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.716482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.716506 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.716637 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:46.716888 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.718179 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.719361 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.720151 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.720235 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.720334 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.720542 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.720589 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260620 11:41:46.721012 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.715914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:38039 (local address 127.0.0.1:54214)
0620 11:41:46.716608 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.716617 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.716660 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.716940 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.716941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.716943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:46.717012 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.717014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.718394 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.718395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.719439 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.719441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.720419 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.720419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.720474 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.720478 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.720525 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.720760 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":196,"thread_start_us":141,"threads_started":1}
I20260620 11:41:46.721200 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.716092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.5.122.131:38039)
0620 11:41:46.716500 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.716506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.716719 (+ 213us) server_negotiation.cc:407] Connection header received
0620 11:41:46.716770 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.716772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.716789 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.716863 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:46.717176 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.718309 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.719526 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.720258 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.720317 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.720389 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.720569 (+ 180us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.720851 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":87,"threads_started":1}
I20260620 11:41:46.721127 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.715914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:33263 (local address 127.0.0.1:38724)
0620 11:41:46.716141 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.716209 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.716289 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.716688 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.716692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.716695 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:46.716780 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.716782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.718283 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.718284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.719265 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.719268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.720373 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.720374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.720440 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.720445 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.721000 (+ 555us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.721049 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":2,"thread_start_us":143,"threads_started":1}
I20260620 11:41:46.721779 23256 tablet_service.cc:1467] Processing CreateTablet for tablet 0d944ede632e4d0dbb889384e8002e55 (DEFAULT_TABLE table=test-table [id=1329a5a8972f42639200803531e1c921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:46.722255 23256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d944ede632e4d0dbb889384e8002e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:46.722285 23518 tablet_service.cc:1467] Processing CreateTablet for tablet 0d944ede632e4d0dbb889384e8002e55 (DEFAULT_TABLE table=test-table [id=1329a5a8972f42639200803531e1c921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:46.722605 23518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d944ede632e4d0dbb889384e8002e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:46.722620 23387 tablet_service.cc:1467] Processing CreateTablet for tablet 0d944ede632e4d0dbb889384e8002e55 (DEFAULT_TABLE table=test-table [id=1329a5a8972f42639200803531e1c921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:46.722882 23387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d944ede632e4d0dbb889384e8002e55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:46.725711 23598 tablet_bootstrap.cc:492] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: Bootstrap starting.
I20260620 11:41:46.726176 23599 tablet_bootstrap.cc:492] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Bootstrap starting.
I20260620 11:41:46.726231 23600 tablet_bootstrap.cc:492] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7: Bootstrap starting.
I20260620 11:41:46.726541 23598 tablet_bootstrap.cc:654] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:46.726850 23599 tablet_bootstrap.cc:654] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:46.726935 23600 tablet_bootstrap.cc:654] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:46.726912 23598 log.cc:824] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:46.727223 23599 log.cc:824] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:46.727329 23600 log.cc:824] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:46.727960 23599 tablet_bootstrap.cc:492] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: No bootstrap required, opened a new log
I20260620 11:41:46.728025 23600 tablet_bootstrap.cc:492] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7: No bootstrap required, opened a new log
I20260620 11:41:46.727980 23598 tablet_bootstrap.cc:492] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: No bootstrap required, opened a new log
I20260620 11:41:46.728021 23599 ts_tablet_manager.cc:1397] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:41:46.728101 23600 ts_tablet_manager.cc:1397] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:41:46.728168 23598 ts_tablet_manager.cc:1397] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:41:46.730254 23599 raft_consensus.cc:348] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.730433 23599 raft_consensus.cc:374] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:46.730480 23599 raft_consensus.cc:729] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afc471bf38094277873d1130f9851b9c, State: Initialized, Role: FOLLOWER
I20260620 11:41:46.730602 23599 consensus_queue.cc:260] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.730926 23599 ts_tablet_manager.cc:1428] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260620 11:41:46.730958 23584 heartbeater.cc:503] Master 127.5.122.190:46005 was elected leader, sending a full tablet report...
I20260620 11:41:46.730890 23600 raft_consensus.cc:348] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.731074 23600 raft_consensus.cc:374] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:46.730890 23598 raft_consensus.cc:348] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.731122 23600 raft_consensus.cc:729] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8e53e4baf0451683ad2b023ac360c7, State: Initialized, Role: FOLLOWER
I20260620 11:41:46.731158 23598 raft_consensus.cc:374] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:46.731201 23598 raft_consensus.cc:729] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e469aede980f4f31a521142c12c501a4, State: Initialized, Role: FOLLOWER
I20260620 11:41:46.731309 23600 consensus_queue.cc:260] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.731309 23598 consensus_queue.cc:260] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.731590 23598 ts_tablet_manager.cc:1428] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:46.731658 23322 heartbeater.cc:503] Master 127.5.122.190:46005 was elected leader, sending a full tablet report...
I20260620 11:41:46.731714 23453 heartbeater.cc:503] Master 127.5.122.190:46005 was elected leader, sending a full tablet report...
I20260620 11:41:46.731590 23600 ts_tablet_manager.cc:1428] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 11:41:46.742897 23606 raft_consensus.cc:482] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:46.743081 23606 raft_consensus.cc:504] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.743523 23606 leader_election.cc:288] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd8e53e4baf0451683ad2b023ac360c7 (127.5.122.129:46373), afc471bf38094277873d1130f9851b9c (127.5.122.131:38039)
I20260620 11:41:46.746739 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.743811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:33605 (local address 127.5.122.131:38039)
0620 11:41:46.743867 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.743870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.744002 (+ 132us) server_negotiation.cc:407] Connection header received
0620 11:41:46.744146 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.744147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.744165 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.744251 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:46.744436 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.744977 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.745937 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.746343 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.746444 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.746453 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.746611 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.746663 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 11:41:46.746739 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.743839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:53483 (local address 127.5.122.129:46373)
0620 11:41:46.743890 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.743892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.743902 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:41:46.743936 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.743937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.743949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.744025 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:46.744237 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.744890 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.745890 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.746343 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.746433 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.746444 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.746601 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.746644 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260620 11:41:46.746745 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.743735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46373 (local address 127.5.122.130:53483)
0620 11:41:46.743804 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.743819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.743873 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.744030 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.744030 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.744032 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:46.744142 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.744144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.744988 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.744989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.745779 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.745782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.746431 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.746432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.746511 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.746515 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.746592 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.746644 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260620 11:41:46.746758 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.743751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:38039 (local address 127.5.122.130:33605)
0620 11:41:46.743930 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.743938 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:46.744016 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:46.744279 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:46.744280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:46.744282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:46.744345 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.744347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.745067 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.745069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.745836 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:46.745839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.746446 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.746447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:46.746525 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.746528 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:46.746603 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 11:41:46.746647 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260620 11:41:46.747143 23538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d944ede632e4d0dbb889384e8002e55" candidate_uuid: "e469aede980f4f31a521142c12c501a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afc471bf38094277873d1130f9851b9c" is_pre_election: true
I20260620 11:41:46.747339 23538 raft_consensus.cc:2454] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e469aede980f4f31a521142c12c501a4 in term 0.
I20260620 11:41:46.747604 23340 leader_election.cc:302] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afc471bf38094277873d1130f9851b9c, e469aede980f4f31a521142c12c501a4; no voters:
I20260620 11:41:46.747596 23276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d944ede632e4d0dbb889384e8002e55" candidate_uuid: "e469aede980f4f31a521142c12c501a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd8e53e4baf0451683ad2b023ac360c7" is_pre_election: true
I20260620 11:41:46.747763 23276 raft_consensus.cc:2454] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e469aede980f4f31a521142c12c501a4 in term 0.
I20260620 11:41:46.747750 23606 raft_consensus.cc:2781] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:41:46.747830 23606 raft_consensus.cc:482] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:41:46.747861 23606 raft_consensus.cc:3037] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:46.749076 23606 raft_consensus.cc:504] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.749330 23606 leader_election.cc:288] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [CANDIDATE]: Term 1 election: Requested vote from peers cd8e53e4baf0451683ad2b023ac360c7 (127.5.122.129:46373), afc471bf38094277873d1130f9851b9c (127.5.122.131:38039)
I20260620 11:41:46.749469 23538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d944ede632e4d0dbb889384e8002e55" candidate_uuid: "e469aede980f4f31a521142c12c501a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afc471bf38094277873d1130f9851b9c"
I20260620 11:41:46.749573 23538 raft_consensus.cc:3037] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:46.750820 23538 raft_consensus.cc:2454] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e469aede980f4f31a521142c12c501a4 in term 1.
I20260620 11:41:46.751039 23340 leader_election.cc:302] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afc471bf38094277873d1130f9851b9c, e469aede980f4f31a521142c12c501a4; no voters:
I20260620 11:41:46.751150 23606 raft_consensus.cc:2781] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:46.751323 23606 raft_consensus.cc:686] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 1 LEADER]: Becoming Leader. State: Replica: e469aede980f4f31a521142c12c501a4, State: Running, Role: LEADER
I20260620 11:41:46.751456 23606 consensus_queue.cc:237] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } }
I20260620 11:41:46.752581 23133 catalog_manager.cc:5495] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e469aede980f4f31a521142c12c501a4 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "e469aede980f4f31a521142c12c501a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e469aede980f4f31a521142c12c501a4" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 33263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 } health_report { overall_health: UNKNOWN } } }
I20260620 11:41:46.753010 23276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d944ede632e4d0dbb889384e8002e55" candidate_uuid: "e469aede980f4f31a521142c12c501a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd8e53e4baf0451683ad2b023ac360c7"
I20260620 11:41:46.753114 23276 raft_consensus.cc:3037] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:46.754396 23276 raft_consensus.cc:2454] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e469aede980f4f31a521142c12c501a4 in term 1.
I20260620 11:41:46.758579 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.756255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.5.122.130:33263)
0620 11:41:46.756299 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:46.756302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:46.756311 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:41:46.756360 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:46.756360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:46.756380 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:46.756456 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:46.756552 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.757098 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.757829 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:46.758123 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:46.758184 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:46.758304 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:46.758408 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:41:46.758473 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:46.758488 (+ 15us) server_negotiation.cc:299] Negotiation successful
0620 11:41:46.758504 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:41:46.760739 23538 raft_consensus.cc:1261] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 1 FOLLOWER]: Refusing update from remote peer e469aede980f4f31a521142c12c501a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:46.760780 23276 raft_consensus.cc:1261] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 1 FOLLOWER]: Refusing update from remote peer e469aede980f4f31a521142c12c501a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:46.761158 23606 consensus_queue.cc:1035] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afc471bf38094277873d1130f9851b9c" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 38039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:46.761337 23608 consensus_queue.cc:1035] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd8e53e4baf0451683ad2b023ac360c7" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 46373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:46.765597 23612 mvcc.cc:205] Tried to move back new op lower bound from 7298890574888906752 to 7298890574855131136. Current Snapshot: MvccSnapshot[applied={T|T < 7298890574888906752 or (T in {7298890574888906752})}]
I20260620 11:41:46.766870 23133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:46.766993 23133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:46.767827 23133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b185e16b06624132b6091dedfffcf2ac: Sending DeleteTablet for 3 replicas of tablet 0d944ede632e4d0dbb889384e8002e55
I20260620 11:41:46.768204 23387 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d944ede632e4d0dbb889384e8002e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:46 UTC) from {username='slave'} at 127.0.0.1:38724
I20260620 11:41:46.768200 23256 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d944ede632e4d0dbb889384e8002e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:46 UTC) from {username='slave'} at 127.0.0.1:45248
I20260620 11:41:46.768443 23620 tablet_replica.cc:330] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7: stopping tablet replica
I20260620 11:41:46.768445 23518 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d944ede632e4d0dbb889384e8002e55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:46 UTC) from {username='slave'} at 127.0.0.1:54214
I20260620 11:41:46.768558 23620 raft_consensus.cc:2229] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:46.768704 23620 raft_consensus.cc:2258] T 0d944ede632e4d0dbb889384e8002e55 P cd8e53e4baf0451683ad2b023ac360c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:46.768707 23622 tablet_replica.cc:330] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: stopping tablet replica
I20260620 11:41:46.768959 23622 raft_consensus.cc:2229] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:46.769071 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 23194
I20260620 11:41:46.770201 23622 raft_consensus.cc:2258] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:46.770761 23622 ts_tablet_manager.cc:1905] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:46.771436 23621 tablet_replica.cc:330] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: stopping tablet replica
I20260620 11:41:46.771555 23621 raft_consensus.cc:2229] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:41:46.771679 23621 raft_consensus.cc:2258] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:46.773052 23622 ts_tablet_manager.cc:1918] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:46.773149 23622 log.cc:1196] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/0d944ede632e4d0dbb889384e8002e55
I20260620 11:41:46.773450 23622 ts_tablet_manager.cc:1939] T 0d944ede632e4d0dbb889384e8002e55 P afc471bf38094277873d1130f9851b9c: Deleting consensus metadata
I20260620 11:41:46.774102 23118 catalog_manager.cc:4841] TS afc471bf38094277873d1130f9851b9c (127.5.122.131:38039): tablet 0d944ede632e4d0dbb889384e8002e55 (table test-table [id=1329a5a8972f42639200803531e1c921]) successfully deleted
I20260620 11:41:46.774482 23621 ts_tablet_manager.cc:1905] T 0d944ede632e4d0dbb889384e8002e55 P e469aede980f4f31a521142c12c501a4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260620 11:41:46.775580 23119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:46373 (error 108)
I20260620 11:41:46.776211 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 23325
I20260620 11:41:46.776459 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.776309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:46373 (local address 127.0.0.1:45258)
0620 11:41:46.776368 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.776413 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 11:41:46.776578 23119 catalog_manager.cc:4584] TS cd8e53e4baf0451683ad2b023ac360c7 (127.5.122.129:46373): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d944ede632e4d0dbb889384e8002e55: Network error: Client connection negotiation failed: client connection to 127.5.122.129:46373: connect: Connection refused (error 111)
I20260620 11:41:46.782414 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 23456
I20260620 11:41:46.782497 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:46.782351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:33263 (local address 127.0.0.1:38748)
0620 11:41:46.782418 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:46.782454 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.130:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 11:41:46.787506 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 23103
2026-06-20T11:41:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (713 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:46 dist-test-slave-szdr krb5kdc[23626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:46 dist-test-slave-szdr krb5kdc[23626](info): set up 2 sockets
Jun 20 11:41:46 dist-test-slave-szdr krb5kdc[23626](info): commencing operation
krb5kdc: starting...
W20260620 11:41:47.837919 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:47 dist-test-slave-szdr krb5kdc[23626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:41:47.972652 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38785
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:38785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:48.074652 23639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:48.074954 23639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:48.074996 23639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:48.076787 23639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:48.076867 23639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:48.076890 23639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:48.076912 23639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:48.076939 23639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:48.079202 23639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:38785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:38785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:48.079614 23639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:48.079921 23639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:48.085289 23645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.085346 23644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.085321 23647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:48.085407 23639 server_base.cc:1034] running on GCE node
I20260620 11:41:48.085695 23639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:48.086170 23639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:48.087359 23639 hybrid_clock.cc:648] HybridClock initialized: now 1781955708087321 us; error 50 us; skew 500 ppm
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955708, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:48.090175 23639 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:41:48.090751 23639 webserver.cc:460] Webserver started at http://127.5.122.190:36889/ using document root <none> and password file <none>
I20260620 11:41:48.091027 23639 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:48.091100 23639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:48.091271 23639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:48.092429 23639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "9688f3a7458747a4ba416226f1d68f2b"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "a99e0bf730808d9460095ecfd9d9054a"
server_key_iv: "26da7797084fd4e0554dd74dc217f011"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.092819 23639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "9688f3a7458747a4ba416226f1d68f2b"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "a99e0bf730808d9460095ecfd9d9054a"
server_key_iv: "26da7797084fd4e0554dd74dc217f011"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.094394 23639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:41:48.095248 23654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.095468 23639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:48.095597 23639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "9688f3a7458747a4ba416226f1d68f2b"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "a99e0bf730808d9460095ecfd9d9054a"
server_key_iv: "26da7797084fd4e0554dd74dc217f011"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.095707 23639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:48.111195 23639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:48.114089 23639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:48.114286 23639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:48.118620 23639 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:38785
I20260620 11:41:48.118656 23706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:38785 every 8 connection(s)
I20260620 11:41:48.119097 23639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:48.120060 23707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:48.122582 23707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b: Bootstrap starting.
I20260620 11:41:48.123312 23707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:48.123647 23707 log.cc:824] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:48.124465 23707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b: No bootstrap required, opened a new log
I20260620 11:41:48.126361 23707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9688f3a7458747a4ba416226f1d68f2b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38785 } }
I20260620 11:41:48.126541 23707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:48.126566 23707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9688f3a7458747a4ba416226f1d68f2b, State: Initialized, Role: FOLLOWER
I20260620 11:41:48.126791 23707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9688f3a7458747a4ba416226f1d68f2b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38785 } }
I20260620 11:41:48.126875 23707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:48.126905 23707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:48.126987 23707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:48.127636 23707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9688f3a7458747a4ba416226f1d68f2b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38785 } }
I20260620 11:41:48.127735 23707 leader_election.cc:302] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9688f3a7458747a4ba416226f1d68f2b; no voters:
I20260620 11:41:48.127949 23707 leader_election.cc:288] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:48.128014 23710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:48.128283 23707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:48.128283 23710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [term 1 LEADER]: Becoming Leader. State: Replica: 9688f3a7458747a4ba416226f1d68f2b, State: Running, Role: LEADER
I20260620 11:41:48.128468 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 23639
I20260620 11:41:48.128536 23710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9688f3a7458747a4ba416226f1d68f2b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38785 } }
I20260620 11:41:48.128587 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:48.128819 5610 external_mini_cluster.cc:1428] Setting key 83b421dd1aaaa7be4a2374e5f3f32f60
I20260620 11:41:48.128929 23712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9688f3a7458747a4ba416226f1d68f2b. Latest consensus state: current_term: 1 leader_uuid: "9688f3a7458747a4ba416226f1d68f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9688f3a7458747a4ba416226f1d68f2b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38785 } } }
I20260620 11:41:48.129016 23712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:48.129096 23711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9688f3a7458747a4ba416226f1d68f2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9688f3a7458747a4ba416226f1d68f2b" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 38785 } } }
I20260620 11:41:48.129175 23711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b [sys.catalog]: This master's current role is: LEADER
W20260620 11:41:48.130321 23727 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:41:48.130385 23727 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:41:48.130493 23719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:48.130730 23719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:48.132594 23719 catalog_manager.cc:1347] Generated new cluster ID: 1c2df42c251445b0bfbe10d27c1e56d0
I20260620 11:41:48.132663 23719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:48.140874 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.129500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.5.122.190:38785)
0620 11:41:48.129690 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.129701 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.130129 (+ 428us) server_negotiation.cc:407] Connection header received
0620 11:41:48.130271 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.130279 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.130342 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.130493 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:48.131426 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.132579 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.133420 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.133922 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.134097 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.136867 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:48.136872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:48.136873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:48.136887 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:48.139068 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.139558 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.139561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.139562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.139673 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.139899 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.139900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.139900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.140165 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:48.140335 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.140441 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.140689 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260620 11:41:48.162284 23719 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:48.163079 23719 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:48.166679 23719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b: Generated new TSK 0
I20260620 11:41:48.166884 23719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:41:48.203145 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38785
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:48.290499 23733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:48.290796 23733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:48.290843 23733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:48.292167 23733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:48.292269 23733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:48.292321 23733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:48.294086 23733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:48.294435 23733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:48.294723 23733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:48.300668 23741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.300668 23738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.300683 23739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:48.300705 23733 server_base.cc:1034] running on GCE node
I20260620 11:41:48.301174 23733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:48.301498 23733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:48.302677 23733 hybrid_clock.cc:648] HybridClock initialized: now 1781955708302663 us; error 34 us; skew 500 ppm
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955708, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:48.305529 23733 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:41:48.306145 23733 webserver.cc:460] Webserver started at http://127.5.122.129:41829/ using document root <none> and password file <none>
I20260620 11:41:48.306422 23733 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:48.306500 23733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:48.306675 23733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:48.307813 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "518faf465007436c8cfedfaa118f2f0d"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "172abe429b58fb87de2dbcf1d7be6749"
server_key_iv: "ac4622570cd0abd129886b2835bd0463"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.308199 23733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "518faf465007436c8cfedfaa118f2f0d"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "172abe429b58fb87de2dbcf1d7be6749"
server_key_iv: "ac4622570cd0abd129886b2835bd0463"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.309679 23733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:41:48.310650 23748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.310847 23733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:48.310966 23733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "518faf465007436c8cfedfaa118f2f0d"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "172abe429b58fb87de2dbcf1d7be6749"
server_key_iv: "ac4622570cd0abd129886b2835bd0463"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.311054 23733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:48.331210 23733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:48.333604 23733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:48.333801 23733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:48.334102 23733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:48.334460 23733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:48.334518 23733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.334561 23733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:48.334589 23733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.340729 23733 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:45013
I20260620 11:41:48.340790 23861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:45013 every 8 connection(s)
I20260620 11:41:48.341173 23733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955708, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:48.349550 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 23733
I20260620 11:41:48.349759 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:41:48.350045 5610 external_mini_cluster.cc:1428] Setting key 3d009468b172d1adf40796dbfd944d63
I20260620 11:41:48.351807 23864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.342048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38785 (local address 127.5.122.129:37581)
0620 11:41:48.342234 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.342275 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.342826 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.343182 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.343186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.343798 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:48.344016 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.344021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.344760 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.344761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.345605 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.345608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.346068 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.346069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.346207 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.346735 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:48.346743 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:48.348273 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:48.350209 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:48.350220 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:48.350224 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:48.350556 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:48.350792 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:48.350793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:48.350793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:48.350943 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:48.351242 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:48.351247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:48.351387 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.351480 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.351622 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260620 11:41:48.351797 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.342051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:37581 (local address 127.5.122.190:38785)
0620 11:41:48.342118 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.342120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.342366 (+ 246us) server_negotiation.cc:407] Connection header received
0620 11:41:48.343013 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.343015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.343029 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.343086 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:48.344128 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.344653 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.345705 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.345915 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.346052 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.348403 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:48.348405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:48.348405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:48.348429 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:48.350069 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.350630 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.350631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.350632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.350710 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.351005 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.351006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.351006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.351173 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:48.351246 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.351541 (+ 295us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.351655 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:41:48.352841 23862 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38785
I20260620 11:41:48.352985 23862 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:48.353216 23862 heartbeater.cc:511] Master 127.5.122.190:38785 requested a full tablet report, sending...
I20260620 11:41:48.353803 23671 ts_manager.cc:194] Registered new tserver with Master: 518faf465007436c8cfedfaa118f2f0d (127.5.122.129:45013)
I20260620 11:41:48.354508 23671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:37581
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:41:48.377452 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38785
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:48.464599 23867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:48.464913 23867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:48.464959 23867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:48.466385 23867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:48.466457 23867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:48.466508 23867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:48.468187 23867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:48.468497 23867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:48.468771 23867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:48.474289 23872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.474319 23873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.474291 23875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:48.474403 23867 server_base.cc:1034] running on GCE node
I20260620 11:41:48.474792 23867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:48.475035 23867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:48.476181 23867 hybrid_clock.cc:648] HybridClock initialized: now 1781955708476169 us; error 30 us; skew 500 ppm
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955708, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:48.478984 23867 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:41:48.479504 23867 webserver.cc:460] Webserver started at http://127.5.122.130:44431/ using document root <none> and password file <none>
I20260620 11:41:48.479707 23867 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:48.479753 23867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:48.479851 23867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:48.480943 23867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "1a7d3552154648b9b2ddd47de90582f2"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "32d37d5c45e4f9c60b505821d680f55f"
server_key_iv: "76a3ceb73281ee97881a2635910a5882"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.481274 23867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "1a7d3552154648b9b2ddd47de90582f2"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "32d37d5c45e4f9c60b505821d680f55f"
server_key_iv: "76a3ceb73281ee97881a2635910a5882"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.482816 23867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:41:48.483579 23882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.483817 23867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:48.483892 23867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "1a7d3552154648b9b2ddd47de90582f2"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "32d37d5c45e4f9c60b505821d680f55f"
server_key_iv: "76a3ceb73281ee97881a2635910a5882"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.483953 23867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:48.498960 23867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:48.502141 23867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:48.502329 23867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:48.502575 23867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:48.502980 23867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:48.503023 23867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.503050 23867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:48.503064 23867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.509573 23867 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:36477
I20260620 11:41:48.509655 23995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:36477 every 8 connection(s)
I20260620 11:41:48.510128 23867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:48.513103 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 23867
I20260620 11:41:48.513262 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:41:48.513512 5610 external_mini_cluster.cc:1428] Setting key 18f957766fced3ec217a720bfcaadf75
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955708, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:48.521569 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.510975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:58779 (local address 127.5.122.190:38785)
0620 11:41:48.511042 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.511044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.511248 (+ 204us) server_negotiation.cc:407] Connection header received
0620 11:41:48.511817 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.511819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.511831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.511902 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:48.512950 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.513884 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.515147 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.515447 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.515524 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.518370 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:48.518372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:48.518373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:48.518383 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:48.519866 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.520452 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.520452 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.520453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.520504 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.520814 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.520815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.520815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.520943 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:48.520993 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.521413 (+ 420us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.521497 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:41:48.521876 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.510929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38785 (local address 127.5.122.130:58779)
0620 11:41:48.511102 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.511147 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.511664 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.511983 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.511987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.512612 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:48.512821 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.512827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.514055 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.514056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.515043 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.515046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.515537 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.515538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.515708 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.516323 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:48.516335 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:48.518260 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:48.519981 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:48.519991 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:48.519994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:48.520356 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:48.520572 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:48.520573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:48.520574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:48.520745 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:48.521010 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:48.521018 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:48.521219 (+ 201us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.521378 (+ 159us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.521598 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260620 11:41:48.522506 23996 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38785
I20260620 11:41:48.522605 23996 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:48.522823 23996 heartbeater.cc:511] Master 127.5.122.190:38785 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
I20260620 11:41:48.523258 23671 ts_manager.cc:194] Registered new tserver with Master: 1a7d3552154648b9b2ddd47de90582f2 (127.5.122.130:36477)
I20260620 11:41:48.523653 23671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:58779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:41:48.541429 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38785
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:48.629281 24001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:48.629509 24001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:48.629535 24001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:48.630892 24001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:48.630949 24001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:48.630980 24001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:48.632524 24001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:38785
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:48.632783 24001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:48.633029 24001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:48.638286 24007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.638347 24009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:48.638307 24006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:48.638438 24001 server_base.cc:1034] running on GCE node
I20260620 11:41:48.638731 24001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:48.639000 24001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:48.640172 24001 hybrid_clock.cc:648] HybridClock initialized: now 1781955708640156 us; error 47 us; skew 500 ppm
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955708, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:48.643056 24001 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:41:48.643527 24001 webserver.cc:460] Webserver started at http://127.5.122.131:40609/ using document root <none> and password file <none>
I20260620 11:41:48.643776 24001 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:48.643849 24001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:48.644001 24001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:48.645126 24001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "2dc6496452c143b7aff2a6b43e473681"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "a1aaccf4c03144e0dfb07c09676dd91d"
server_key_iv: "1dd7f8cbf9e8777e3240cc776037cd4d"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.645509 24001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "2dc6496452c143b7aff2a6b43e473681"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "a1aaccf4c03144e0dfb07c09676dd91d"
server_key_iv: "1dd7f8cbf9e8777e3240cc776037cd4d"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.647051 24001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:48.647907 24016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.648123 24001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:48.648236 24001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "2dc6496452c143b7aff2a6b43e473681"
format_stamp: "Formatted at 2026-06-20 11:41:48 on dist-test-slave-szdr"
server_key: "a1aaccf4c03144e0dfb07c09676dd91d"
server_key_iv: "1dd7f8cbf9e8777e3240cc776037cd4d"
server_key_version: "encryptionkey@0"
I20260620 11:41:48.648319 24001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:48.661753 24001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:48.664033 24001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:48.664233 24001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:48.664512 24001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:48.664880 24001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:48.664940 24001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.664978 24001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:48.665024 24001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:48.671487 24001 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:34347
I20260620 11:41:48.671545 24129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:34347 every 8 connection(s)
I20260620 11:41:48.671938 24001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:48.677337 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24001
I20260620 11:41:48.677430 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:41:48.677627 5610 external_mini_cluster.cc:1428] Setting key 8b80e6deea1b6ecaf59a56234d47f337
Jun 20 11:41:48 dist-test-slave-szdr krb5kdc[23626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955708, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:48.681885 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.672751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:56399 (local address 127.5.122.190:38785)
0620 11:41:48.672827 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.672830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.673037 (+ 207us) server_negotiation.cc:407] Connection header received
0620 11:41:48.673589 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.673590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.673601 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.673658 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:48.674712 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.675217 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.676236 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.676433 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.676487 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.679194 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:48.679195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:48.679196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:48.679203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:48.680615 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.681022 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.681023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.681024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.681076 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.681298 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.681299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.681299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.681424 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:48.681480 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.681750 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.681815 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:41:48.682224 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.672720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:38785 (local address 127.5.122.131:56399)
0620 11:41:48.672920 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.672961 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.673490 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.673716 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.673719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.674410 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:48.674620 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.674626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.675319 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.675321 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.676147 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.676149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.676511 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.676512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.676622 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.677244 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:48.677253 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:48.679112 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:48.680694 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:48.680701 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:48.680703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:48.680953 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:48.681135 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:48.681136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:48.681136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:48.681243 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:48.681480 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:48.681484 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:48.681608 (+ 124us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.681734 (+ 126us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.681964 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260620 11:41:48.682785 24130 heartbeater.cc:347] Connected to a master server at 127.5.122.190:38785
I20260620 11:41:48.682880 24130 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:48.683070 24130 heartbeater.cc:511] Master 127.5.122.190:38785 requested a full tablet report, sending...
I20260620 11:41:48.683519 23671 ts_manager.cc:194] Registered new tserver with Master: 2dc6496452c143b7aff2a6b43e473681 (127.5.122.131:34347)
I20260620 11:41:48.683909 23671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:56399
I20260620 11:41:48.689644 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:41:48.697643 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.691312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.5.122.190:38785)
0620 11:41:48.691379 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.691381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.691435 (+ 54us) server_negotiation.cc:407] Connection header received
0620 11:41:48.691534 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.691535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.691544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.691609 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:48.692201 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.692696 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.693411 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.693593 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.693646 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.695095 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:48.695096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:48.695097 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:48.695104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:48.696455 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.696931 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.696932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.696932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.696986 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:48.697177 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:48.697178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:48.697178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:48.697322 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:48.697418 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.697518 (+ 100us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.697568 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:41:48.699528 23671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:41:48.701303 23671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:41:48.708475 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.704083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.5.122.129:45013)
0620 11:41:48.704830 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.704835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.704845 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:41:48.704886 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.704888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.704905 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.704978 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:48.705175 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.706291 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.707504 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.708107 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.708174 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.708230 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.708349 (+ 119us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.708390 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":86,"threads_started":1}
I20260620 11:41:48.708484 24142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.703945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:45013 (local address 127.0.0.1:58116)
0620 11:41:48.704093 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.704141 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.704216 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.705065 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.705066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.705068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:48.705107 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.705108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.706398 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.706399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.707393 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.707395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.708194 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.708195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.708261 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.708266 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.708320 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.708381 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":23,"thread_start_us":69,"threads_started":1}
I20260620 11:41:48.708575 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.704130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.5.122.131:34347)
0620 11:41:48.704496 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.704502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.704516 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:41:48.704561 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.704564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.704582 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.704667 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:48.704901 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.705725 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.707147 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.707978 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.708058 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.708120 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.708438 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.708488 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":99,"threads_started":1}
I20260620 11:41:48.708726 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.703949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:34347 (local address 127.0.0.1:56842)
0620 11:41:48.704298 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.704309 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.704366 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.704694 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.704697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.704700 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:48.704786 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.704789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.705825 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.705826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.707051 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.707054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.708071 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.708072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.708169 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.708187 (+ 18us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.708260 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.708360 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":88,"threads_started":1}
I20260620 11:41:48.708834 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.704457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.5.122.130:36477)
0620 11:41:48.704664 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.704669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.704681 (+ 12us) server_negotiation.cc:407] Connection header received
0620 11:41:48.704728 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.704742 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.704760 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.704839 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:48.705079 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.706235 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.707618 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.708348 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.708409 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.708457 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.708703 (+ 246us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.708749 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260620 11:41:48.708972 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.704170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:36477 (local address 127.0.0.1:52790)
0620 11:41:48.704296 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.704307 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.704363 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.704947 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.704948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.704950 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:48.704994 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.704996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.706353 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.706354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.707516 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.707519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.708491 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.708492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.708551 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.708555 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.708606 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.708648 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260620 11:41:48.709656 24064 tablet_service.cc:1467] Processing CreateTablet for tablet 3c91b389783b4d7b866314dc26be2be1 (DEFAULT_TABLE table=test-table [id=c70fc97228ed4cd89ea14ef0dd194d45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:48.709733 23930 tablet_service.cc:1467] Processing CreateTablet for tablet 3c91b389783b4d7b866314dc26be2be1 (DEFAULT_TABLE table=test-table [id=c70fc97228ed4cd89ea14ef0dd194d45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:48.709925 23796 tablet_service.cc:1467] Processing CreateTablet for tablet 3c91b389783b4d7b866314dc26be2be1 (DEFAULT_TABLE table=test-table [id=c70fc97228ed4cd89ea14ef0dd194d45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:48.710099 24064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c91b389783b4d7b866314dc26be2be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:48.710117 23930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c91b389783b4d7b866314dc26be2be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:48.710258 23796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c91b389783b4d7b866314dc26be2be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:48.713101 24147 tablet_bootstrap.cc:492] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Bootstrap starting.
I20260620 11:41:48.713567 24148 tablet_bootstrap.cc:492] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Bootstrap starting.
I20260620 11:41:48.713900 24149 tablet_bootstrap.cc:492] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: Bootstrap starting.
I20260620 11:41:48.713981 24147 tablet_bootstrap.cc:654] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:48.714478 24148 tablet_bootstrap.cc:654] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:48.714489 24149 tablet_bootstrap.cc:654] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:48.714602 24147 log.cc:824] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:48.715050 24148 log.cc:824] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:48.715140 24149 log.cc:824] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:48.715581 24147 tablet_bootstrap.cc:492] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: No bootstrap required, opened a new log
I20260620 11:41:48.715663 24147 ts_tablet_manager.cc:1397] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:41:48.715981 24148 tablet_bootstrap.cc:492] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: No bootstrap required, opened a new log
I20260620 11:41:48.716077 24148 ts_tablet_manager.cc:1397] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260620 11:41:48.717221 24147 raft_consensus.cc:348] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.717383 24147 raft_consensus.cc:374] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:48.717408 24147 raft_consensus.cc:729] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a7d3552154648b9b2ddd47de90582f2, State: Initialized, Role: FOLLOWER
I20260620 11:41:48.717545 24147 consensus_queue.cc:260] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.717540 24148 raft_consensus.cc:348] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.717752 24148 raft_consensus.cc:374] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:48.717808 24148 raft_consensus.cc:729] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc6496452c143b7aff2a6b43e473681, State: Initialized, Role: FOLLOWER
I20260620 11:41:48.717883 24147 ts_tablet_manager.cc:1428] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 11:41:48.717936 23996 heartbeater.cc:503] Master 127.5.122.190:38785 was elected leader, sending a full tablet report...
I20260620 11:41:48.717939 24148 consensus_queue.cc:260] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.718284 24148 ts_tablet_manager.cc:1428] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 11:41:48.718322 24130 heartbeater.cc:503] Master 127.5.122.190:38785 was elected leader, sending a full tablet report...
I20260620 11:41:48.718933 24149 tablet_bootstrap.cc:492] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: No bootstrap required, opened a new log
I20260620 11:41:48.719012 24149 ts_tablet_manager.cc:1397] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260620 11:41:48.720813 24149 raft_consensus.cc:348] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.720976 24149 raft_consensus.cc:374] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:48.721000 24149 raft_consensus.cc:729] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518faf465007436c8cfedfaa118f2f0d, State: Initialized, Role: FOLLOWER
I20260620 11:41:48.721148 24149 consensus_queue.cc:260] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.721443 24149 ts_tablet_manager.cc:1428] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 11:41:48.721500 23862 heartbeater.cc:503] Master 127.5.122.190:38785 was elected leader, sending a full tablet report...
W20260620 11:41:48.760797 23997 tablet.cc:2369] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:41:48.842094 23863 tablet.cc:2369] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:41:48.922616 24131 tablet.cc:2369] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:48.952311 24153 raft_consensus.cc:482] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:48.952500 24153 raft_consensus.cc:504] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.952973 24153 leader_election.cc:288] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 518faf465007436c8cfedfaa118f2f0d (127.5.122.129:45013), 2dc6496452c143b7aff2a6b43e473681 (127.5.122.131:34347)
I20260620 11:41:48.956079 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.953184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:34347 (local address 127.5.122.130:36003)
0620 11:41:48.953253 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.953268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.953323 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.953495 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.953496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.953498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:48.953575 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.953577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.954368 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.954369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.955155 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.955158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.955776 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.955777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.955871 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.955874 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.955947 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.955994 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260620 11:41:48.956079 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.953309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:36003 (local address 127.5.122.131:34347)
0620 11:41:48.953367 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.953370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.953381 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:41:48.953412 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.953413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.953422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.953490 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:48.953656 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.954271 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.955275 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.955678 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.955777 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.955787 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.955953 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.955994 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260620 11:41:48.956249 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.953333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:49953 (local address 127.5.122.129:45013)
0620 11:41:48.953398 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.953401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.953514 (+ 113us) server_negotiation.cc:407] Connection header received
0620 11:41:48.953581 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.953583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.953593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.953678 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:48.953887 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.954443 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.955427 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.955811 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.955919 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.955929 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.956102 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.956144 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:41:48.956439 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.953185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:45013 (local address 127.5.122.130:49953)
0620 11:41:48.953439 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:48.953448 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:48.953523 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:48.953685 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:48.953686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:48.953688 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:48.953806 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.953808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.954539 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.954541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.955317 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:48.955319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.955925 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.955926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:48.956007 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.956014 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 11:41:48.956094 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 11:41:48.956137 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":46,"thread_start_us":135,"threads_started":1}
I20260620 11:41:48.956508 24084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c91b389783b4d7b866314dc26be2be1" candidate_uuid: "1a7d3552154648b9b2ddd47de90582f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc6496452c143b7aff2a6b43e473681" is_pre_election: true
I20260620 11:41:48.956740 24084 raft_consensus.cc:2454] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a7d3552154648b9b2ddd47de90582f2 in term 0.
I20260620 11:41:48.956848 23816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c91b389783b4d7b866314dc26be2be1" candidate_uuid: "1a7d3552154648b9b2ddd47de90582f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518faf465007436c8cfedfaa118f2f0d" is_pre_election: true
I20260620 11:41:48.957026 23816 raft_consensus.cc:2454] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a7d3552154648b9b2ddd47de90582f2 in term 0.
I20260620 11:41:48.957062 23886 leader_election.cc:302] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a7d3552154648b9b2ddd47de90582f2, 2dc6496452c143b7aff2a6b43e473681; no voters:
I20260620 11:41:48.957242 24153 raft_consensus.cc:2781] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:41:48.957341 24153 raft_consensus.cc:482] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:41:48.957379 24153 raft_consensus.cc:3037] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:48.958217 24153 raft_consensus.cc:504] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.958384 24153 leader_election.cc:288] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [CANDIDATE]: Term 1 election: Requested vote from peers 518faf465007436c8cfedfaa118f2f0d (127.5.122.129:45013), 2dc6496452c143b7aff2a6b43e473681 (127.5.122.131:34347)
I20260620 11:41:48.958597 23816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c91b389783b4d7b866314dc26be2be1" candidate_uuid: "1a7d3552154648b9b2ddd47de90582f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518faf465007436c8cfedfaa118f2f0d"
I20260620 11:41:48.958639 24084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c91b389783b4d7b866314dc26be2be1" candidate_uuid: "1a7d3552154648b9b2ddd47de90582f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dc6496452c143b7aff2a6b43e473681"
I20260620 11:41:48.958701 23816 raft_consensus.cc:3037] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:48.958743 24084 raft_consensus.cc:3037] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:48.959501 23816 raft_consensus.cc:2454] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a7d3552154648b9b2ddd47de90582f2 in term 1.
I20260620 11:41:48.959502 24084 raft_consensus.cc:2454] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a7d3552154648b9b2ddd47de90582f2 in term 1.
I20260620 11:41:48.959688 23886 leader_election.cc:302] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a7d3552154648b9b2ddd47de90582f2, 2dc6496452c143b7aff2a6b43e473681; no voters:
I20260620 11:41:48.959782 24153 raft_consensus.cc:2781] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:48.959933 24153 raft_consensus.cc:686] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 1 LEADER]: Becoming Leader. State: Replica: 1a7d3552154648b9b2ddd47de90582f2, State: Running, Role: LEADER
I20260620 11:41:48.960049 24153 consensus_queue.cc:237] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } }
I20260620 11:41:48.960988 23670 catalog_manager.cc:5495] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a7d3552154648b9b2ddd47de90582f2 (127.5.122.130). New cstate: current_term: 1 leader_uuid: "1a7d3552154648b9b2ddd47de90582f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a7d3552154648b9b2ddd47de90582f2" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 36477 } health_report { overall_health: HEALTHY } } }
I20260620 11:41:48.993891 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:48.991281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.5.122.130:36477)
0620 11:41:48.991378 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:48.991380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:48.991389 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:41:48.991427 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:48.991428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:48.991454 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:48.991527 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:48.991725 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.992202 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.993007 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:48.993203 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:48.993263 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:48.993467 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:48.993565 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:41:48.993644 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:48.993692 (+ 48us) server_negotiation.cc:299] Negotiation successful
0620 11:41:48.993704 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 11:41:48.995909 23816 raft_consensus.cc:1261] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 1 FOLLOWER]: Refusing update from remote peer 1a7d3552154648b9b2ddd47de90582f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:48.995916 24084 raft_consensus.cc:1261] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 1 FOLLOWER]: Refusing update from remote peer 1a7d3552154648b9b2ddd47de90582f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:48.996309 24153 consensus_queue.cc:1035] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "518faf465007436c8cfedfaa118f2f0d" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 45013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:48.996455 24157 consensus_queue.cc:1035] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc6496452c143b7aff2a6b43e473681" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 34347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:48.999912 24162 mvcc.cc:205] Tried to move back new op lower bound from 7298890584044449792 to 7298890583901454336. Current Snapshot: MvccSnapshot[applied={T|T < 7298890584044449792 or (T in {7298890584044449792})}]
I20260620 11:41:49.001519 23671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:49.001618 23671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:49.002511 23671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9688f3a7458747a4ba416226f1d68f2b: Sending DeleteTablet for 3 replicas of tablet 3c91b389783b4d7b866314dc26be2be1
I20260620 11:41:49.002875 23796 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c91b389783b4d7b866314dc26be2be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:48 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58116
I20260620 11:41:49.002898 24064 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c91b389783b4d7b866314dc26be2be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:48 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:56842
I20260620 11:41:49.002914 23930 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c91b389783b4d7b866314dc26be2be1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:48 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:52790
I20260620 11:41:49.003233 24170 tablet_replica.cc:330] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d: stopping tablet replica
I20260620 11:41:49.003234 24171 tablet_replica.cc:330] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: stopping tablet replica
I20260620 11:41:49.003383 24170 raft_consensus.cc:2229] T 3c91b389783b4d7b866314dc26be2be1 P 518faf465007436c8cfedfaa118f2f0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:49.003436 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 23733
I20260620 11:41:49.003348 24171 raft_consensus.cc:2229] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:41:49.004659 24171 raft_consensus.cc:2258] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:49.004832 24172 tablet_replica.cc:330] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: stopping tablet replica
I20260620 11:41:49.004935 24172 raft_consensus.cc:2229] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:49.005041 24172 raft_consensus.cc:2258] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:49.005460 24171 ts_tablet_manager.cc:1905] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:49.005559 24172 ts_tablet_manager.cc:1905] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:49.007606 24171 ts_tablet_manager.cc:1918] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:49.007707 24171 log.cc:1196] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/3c91b389783b4d7b866314dc26be2be1
I20260620 11:41:49.007990 24171 ts_tablet_manager.cc:1939] T 3c91b389783b4d7b866314dc26be2be1 P 1a7d3552154648b9b2ddd47de90582f2: Deleting consensus metadata
I20260620 11:41:49.008462 24172 ts_tablet_manager.cc:1918] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:49.008548 24172 log.cc:1196] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/3c91b389783b4d7b866314dc26be2be1
I20260620 11:41:49.008514 23655 catalog_manager.cc:4841] TS 1a7d3552154648b9b2ddd47de90582f2 (127.5.122.130:36477): tablet 3c91b389783b4d7b866314dc26be2be1 (table test-table [id=c70fc97228ed4cd89ea14ef0dd194d45]) successfully deleted
I20260620 11:41:49.008827 24172 ts_tablet_manager.cc:1939] T 3c91b389783b4d7b866314dc26be2be1 P 2dc6496452c143b7aff2a6b43e473681: Deleting consensus metadata
I20260620 11:41:49.009251 23658 catalog_manager.cc:4841] TS 2dc6496452c143b7aff2a6b43e473681 (127.5.122.131:34347): tablet 3c91b389783b4d7b866314dc26be2be1 (table test-table [id=c70fc97228ed4cd89ea14ef0dd194d45]) successfully deleted
W20260620 11:41:49.010057 23655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:45013 (error 108)
I20260620 11:41:49.010623 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.010486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:45013 (local address 127.0.0.1:58130)
0620 11:41:49.010545 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.010582 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:45013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 11:41:49.010675 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 23867
W20260620 11:41:49.011930 23655 catalog_manager.cc:4584] TS 518faf465007436c8cfedfaa118f2f0d (127.5.122.129:45013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c91b389783b4d7b866314dc26be2be1: Network error: Client connection negotiation failed: client connection to 127.5.122.129:45013: connect: Connection refused (error 111)
I20260620 11:41:49.016322 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24001
I20260620 11:41:49.021052 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 23639
2026-06-20T11:41:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2233 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T11:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:49Z Disabled control of system clock
I20260620 11:41:49.050352 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41149
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:49.135295 24177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:49.135586 24177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:49.135627 24177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:49.136916 24177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:49.136997 24177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:49.137018 24177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:49.137040 24177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:49.137068 24177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:49.138700 24177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:49.139021 24177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:49.139444 24177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:49.144742 24182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:49.144690 24183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:49.144726 24185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:49.144906 24177 server_base.cc:1034] running on GCE node
I20260620 11:41:49.145320 24177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:49.145663 24177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:49.146853 24177 hybrid_clock.cc:648] HybridClock initialized: now 1781955709146837 us; error 47 us; skew 500 ppm
I20260620 11:41:49.152752 24177 webserver.cc:460] Webserver started at http://127.5.122.190:34755/ using document root <none> and password file <none>
I20260620 11:41:49.153115 24177 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:49.153203 24177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:49.153383 24177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:49.154640 24177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "593a5355f5b2405f94115f05e09e7cd6"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "69d39b8a5455da1acce0ba39163f4ff0"
server_key_iv: "5da48efa300fc6fcb56dcff394b3c36b"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.155036 24177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "593a5355f5b2405f94115f05e09e7cd6"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "69d39b8a5455da1acce0ba39163f4ff0"
server_key_iv: "5da48efa300fc6fcb56dcff394b3c36b"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.156548 24177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:49.157487 24191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.157732 24177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:49.157855 24177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "593a5355f5b2405f94115f05e09e7cd6"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "69d39b8a5455da1acce0ba39163f4ff0"
server_key_iv: "5da48efa300fc6fcb56dcff394b3c36b"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.157941 24177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:49.171376 24177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:49.171772 24177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:49.171940 24177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:49.176391 24177 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:41149
I20260620 11:41:49.176426 24243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:41149 every 8 connection(s)
I20260620 11:41:49.176820 24177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:49.177851 24244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:49.180447 24244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6: Bootstrap starting.
I20260620 11:41:49.181079 24244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:49.181406 24244 log.cc:824] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:49.182241 24244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6: No bootstrap required, opened a new log
I20260620 11:41:49.184008 24244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593a5355f5b2405f94115f05e09e7cd6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41149 } }
I20260620 11:41:49.184172 24244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:49.184196 24244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593a5355f5b2405f94115f05e09e7cd6, State: Initialized, Role: FOLLOWER
I20260620 11:41:49.184391 24244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593a5355f5b2405f94115f05e09e7cd6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41149 } }
I20260620 11:41:49.184458 24244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:49.184500 24244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:49.184552 24244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:49.185149 24244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593a5355f5b2405f94115f05e09e7cd6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41149 } }
I20260620 11:41:49.185235 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24177
I20260620 11:41:49.185243 24244 leader_election.cc:302] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 593a5355f5b2405f94115f05e09e7cd6; no voters:
I20260620 11:41:49.185349 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:49.185499 24244 leader_election.cc:288] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:49.185536 24247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:49.185582 5610 external_mini_cluster.cc:1428] Setting key 43f9b1a07e7ff030e6ca90133c1565da
I20260620 11:41:49.185662 24247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [term 1 LEADER]: Becoming Leader. State: Replica: 593a5355f5b2405f94115f05e09e7cd6, State: Running, Role: LEADER
I20260620 11:41:49.185762 24247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593a5355f5b2405f94115f05e09e7cd6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41149 } }
I20260620 11:41:49.185809 24244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:49.186168 24248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 593a5355f5b2405f94115f05e09e7cd6. Latest consensus state: current_term: 1 leader_uuid: "593a5355f5b2405f94115f05e09e7cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593a5355f5b2405f94115f05e09e7cd6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41149 } } }
I20260620 11:41:49.186261 24249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "593a5355f5b2405f94115f05e09e7cd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593a5355f5b2405f94115f05e09e7cd6" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41149 } } }
I20260620 11:41:49.186360 24248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:49.186375 24249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:49.186738 24258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:49.187870 24258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:49.189564 24258 catalog_manager.cc:1347] Generated new cluster ID: 9d3247784bb9438182c7a4a934e7e262
I20260620 11:41:49.189628 24258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:49.193153 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.186288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.5.122.190:41149)
0620 11:41:49.186510 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.186520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.187023 (+ 503us) server_negotiation.cc:407] Connection header received
0620 11:41:49.187202 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.187210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.187296 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.187437 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:49.187742 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.189341 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.190701 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.191357 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.191491 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.192353 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:49.192357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:49.192357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:49.192572 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:49.192616 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:49.192630 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:49.192699 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.192834 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.192924 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260620 11:41:49.195616 24258 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:49.196448 24258 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:49.204061 24258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6: Generated new TSK 0
I20260620 11:41:49.204339 24258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:49.205538 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41149
--builtin_ntp_servers=127.5.122.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:49.293720 24268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:49.294049 24268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:49.294104 24268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:49.295481 24268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:49.295575 24268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:49.295640 24268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:49.297232 24268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:49.297549 24268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:49.297884 24268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:49.304388 24273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:49.304528 24276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:49.304394 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:49.304622 24268 server_base.cc:1034] running on GCE node
I20260620 11:41:49.305163 24268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:49.305459 24268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:49.306671 24268 hybrid_clock.cc:648] HybridClock initialized: now 1781955709306653 us; error 34 us; skew 500 ppm
I20260620 11:41:49.308192 24268 webserver.cc:460] Webserver started at http://127.5.122.129:35657/ using document root <none> and password file <none>
I20260620 11:41:49.308465 24268 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:49.308544 24268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:49.308704 24268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:49.309873 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "5975d6218a6040b4a9f5a4651096e255"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "4e7661e3de0a0494aa5921dd1074b509"
server_key_iv: "e9f05dbd33883b3496f84364d6fa23af"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.310343 24268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "5975d6218a6040b4a9f5a4651096e255"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "4e7661e3de0a0494aa5921dd1074b509"
server_key_iv: "e9f05dbd33883b3496f84364d6fa23af"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.311908 24268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:49.312829 24282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.313014 24268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:49.313130 24268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "5975d6218a6040b4a9f5a4651096e255"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "4e7661e3de0a0494aa5921dd1074b509"
server_key_iv: "e9f05dbd33883b3496f84364d6fa23af"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.313266 24268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:49.326061 24268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:49.326440 24268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:49.326619 24268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:49.326889 24268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:49.327260 24268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:49.327315 24268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.327353 24268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:49.327379 24268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.333477 24268 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:38069
I20260620 11:41:49.333521 24395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:38069 every 8 connection(s)
I20260620 11:41:49.333884 24268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:41:49.339637 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.334757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:44021 (local address 127.5.122.190:41149)
0620 11:41:49.334843 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.334845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.335096 (+ 251us) server_negotiation.cc:407] Connection header received
0620 11:41:49.335669 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.335670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.335685 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.335744 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:49.336481 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.337012 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.338084 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.338311 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.338393 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.339313 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:49.339314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:49.339315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:49.339320 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:49.339336 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:49.339346 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:49.339409 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.339508 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.339547 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 11:41:49.339932 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.334749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41149 (local address 127.5.122.129:44021)
0620 11:41:49.334962 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.334999 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.335551 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.335834 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.335838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.335846 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:49.336377 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.336382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.337131 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.337132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.337982 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.337984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.338417 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.338418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.338561 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.339158 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:49.339159 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:49.339191 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:49.339215 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:49.339411 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:49.339417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:49.339421 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.339510 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.339622 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260620 11:41:49.340524 24396 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41149
I20260620 11:41:49.340613 24396 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:49.340766 24396 heartbeater.cc:511] Master 127.5.122.190:41149 requested a full tablet report, sending...
I20260620 11:41:49.341291 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24268
I20260620 11:41:49.341405 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:41:49.341387 24208 ts_manager.cc:194] Registered new tserver with Master: 5975d6218a6040b4a9f5a4651096e255 (127.5.122.129:38069)
I20260620 11:41:49.341624 5610 external_mini_cluster.cc:1428] Setting key 645c4bc9f4202ebe80730bf73a5e9f23
I20260620 11:41:49.342304 24208 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.129:44021
I20260620 11:41:49.342875 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41149
--builtin_ntp_servers=127.5.122.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:49.430622 24399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:49.430862 24399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:49.430888 24399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:49.432160 24399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:49.432209 24399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:49.432245 24399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:49.433710 24399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:49.433943 24399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:49.434331 24399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:49.439899 24404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:49.439903 24405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:49.439899 24407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:49.439921 24399 server_base.cc:1034] running on GCE node
I20260620 11:41:49.440412 24399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:49.440698 24399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:49.441874 24399 hybrid_clock.cc:648] HybridClock initialized: now 1781955709441861 us; error 31 us; skew 500 ppm
I20260620 11:41:49.443310 24399 webserver.cc:460] Webserver started at http://127.5.122.130:34645/ using document root <none> and password file <none>
I20260620 11:41:49.443622 24399 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:49.443698 24399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:49.443863 24399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:49.445083 24399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "e214c727844c42958d055518e97e19ea"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "fd292fc389963840ff0a36b984795eb4"
server_key_iv: "1091d9d9f5d407a391de5f235deb7d4f"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.445477 24399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "e214c727844c42958d055518e97e19ea"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "fd292fc389963840ff0a36b984795eb4"
server_key_iv: "1091d9d9f5d407a391de5f235deb7d4f"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.447028 24399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:49.447888 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.448065 24399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:49.448177 24399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "e214c727844c42958d055518e97e19ea"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "fd292fc389963840ff0a36b984795eb4"
server_key_iv: "1091d9d9f5d407a391de5f235deb7d4f"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.448273 24399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:49.462960 24399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:49.463372 24399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:49.463546 24399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:49.463829 24399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:49.464198 24399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:49.464262 24399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.464311 24399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:49.464337 24399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.470674 24399 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:46545
I20260620 11:41:49.470734 24526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:46545 every 8 connection(s)
I20260620 11:41:49.471171 24399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:49.477312 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.472168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:56895 (local address 127.5.122.190:41149)
0620 11:41:49.472297 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.472300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.472527 (+ 227us) server_negotiation.cc:407] Connection header received
0620 11:41:49.473219 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.473221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.473235 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.473286 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:49.474069 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.474614 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.475699 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.475914 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.475993 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.476982 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:49.476984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:49.476984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:49.476991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:49.477010 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:49.477014 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:49.477084 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.477187 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.477224 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 11:41:49.477583 24529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.472121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41149 (local address 127.5.122.130:56895)
0620 11:41:49.472384 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.472424 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.473081 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.473390 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.473395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.473403 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:49.473940 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.473944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.474736 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.474737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.475580 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.475582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.476017 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.476018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.476169 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.476815 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:49.476816 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:49.476850 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:49.476877 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:49.477086 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:49.477093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:49.477096 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.477190 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.477309 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
I20260620 11:41:49.478207 24527 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41149
I20260620 11:41:49.478320 24527 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:49.478329 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24399
I20260620 11:41:49.478440 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:41:49.478566 24527 heartbeater.cc:511] Master 127.5.122.190:41149 requested a full tablet report, sending...
I20260620 11:41:49.478734 5610 external_mini_cluster.cc:1428] Setting key d70305e9a3bc126ad5201c93ae53749e
I20260620 11:41:49.479024 24208 ts_manager.cc:194] Registered new tserver with Master: e214c727844c42958d055518e97e19ea (127.5.122.130:46545)
I20260620 11:41:49.479470 24208 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.130:56895
I20260620 11:41:49.479749 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41149
--builtin_ntp_servers=127.5.122.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:49.565292 24530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:49.565577 24530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:49.565618 24530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:49.566918 24530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:49.566999 24530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:49.567042 24530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:49.568607 24530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:41149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:49.568917 24530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:49.569255 24530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:49.574445 24538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:49.574462 24535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:49.574633 24530 server_base.cc:1034] running on GCE node
W20260620 11:41:49.574716 24536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:49.574952 24530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:49.575172 24530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:49.576352 24530 hybrid_clock.cc:648] HybridClock initialized: now 1781955709576330 us; error 39 us; skew 500 ppm
I20260620 11:41:49.577791 24530 webserver.cc:460] Webserver started at http://127.5.122.131:38369/ using document root <none> and password file <none>
I20260620 11:41:49.578174 24530 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:49.578258 24530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:49.578440 24530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:49.579630 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "96b3b220718b4adb8bfa8e9fb80fa874"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "ff24d72fb671893b6d90ecdd885b0a70"
server_key_iv: "5c8ec508630979a9ff36eb2e035edfbd"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.580035 24530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "96b3b220718b4adb8bfa8e9fb80fa874"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "ff24d72fb671893b6d90ecdd885b0a70"
server_key_iv: "5c8ec508630979a9ff36eb2e035edfbd"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.581633 24530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:41:49.582613 24544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.582795 24530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:41:49.582906 24530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "96b3b220718b4adb8bfa8e9fb80fa874"
format_stamp: "Formatted at 2026-06-20 11:41:49 on dist-test-slave-szdr"
server_key: "ff24d72fb671893b6d90ecdd885b0a70"
server_key_iv: "5c8ec508630979a9ff36eb2e035edfbd"
server_key_version: "encryptionkey@0"
I20260620 11:41:49.582986 24530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:49.601111 24530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:49.601511 24530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:49.601681 24530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:49.601941 24530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:49.602411 24530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:49.602510 24530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.602576 24530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:49.602603 24530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:49.608572 24530 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:32975
I20260620 11:41:49.608618 24657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:32975 every 8 connection(s)
I20260620 11:41:49.608984 24530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:49.614595 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.609811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:60821 (local address 127.5.122.190:41149)
0620 11:41:49.609902 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.609904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.610168 (+ 264us) server_negotiation.cc:407] Connection header received
0620 11:41:49.610690 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.610692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.610704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.610763 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:49.611422 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.611921 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.613025 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.613224 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.613278 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.614293 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:49.614294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:49.614294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:49.614300 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:49.614316 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:49.614320 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:49.614371 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.614443 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.614525 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 11:41:49.614713 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.609764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:41149 (local address 127.5.122.131:60821)
0620 11:41:49.610030 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.610079 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.610600 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.610820 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.610824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.610832 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:49.611346 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.611350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.612041 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.612043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.612919 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.612922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.613307 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.613308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.613431 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.614120 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:49.614120 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:49.614153 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:49.614179 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:49.614385 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:49.614389 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:49.614392 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.614446 (+ 54us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.614539 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":115,"threads_started":1}
I20260620 11:41:49.615361 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24530
I20260620 11:41:49.615444 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:41:49.615679 5610 external_mini_cluster.cc:1428] Setting key d50efd059c5ba31147bac6f7a271205a
I20260620 11:41:49.615697 24658 heartbeater.cc:347] Connected to a master server at 127.5.122.190:41149
I20260620 11:41:49.615813 24658 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:49.616039 24658 heartbeater.cc:511] Master 127.5.122.190:41149 requested a full tablet report, sending...
I20260620 11:41:49.616448 24207 ts_manager.cc:194] Registered new tserver with Master: 96b3b220718b4adb8bfa8e9fb80fa874 (127.5.122.131:32975)
I20260620 11:41:49.616911 24207 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.131:60821
I20260620 11:41:49.627585 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:41:49.632704 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.629273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33908 (local address 127.5.122.190:41149)
0620 11:41:49.629358 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.629360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.629452 (+ 92us) server_negotiation.cc:407] Connection header received
0620 11:41:49.629562 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.629563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.629585 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.629647 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:49.629862 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.630454 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.631276 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.631493 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.631579 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.632346 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:49.632348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:49.632348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:49.632355 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:49.632380 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:49.632384 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:49.632490 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.632588 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.632600 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 11:41:49.634756 24207 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:41:49.636475 24207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:41:49.643555 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.639125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:46545 (local address 127.0.0.1:55570)
0620 11:41:49.639538 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.639584 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.639648 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.639946 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.639949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.639951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:49.640029 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.640032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.641027 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.641029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.642253 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.642256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.643210 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.643211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.643293 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.643308 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.643391 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.643447 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":114,"threads_started":1}
I20260620 11:41:49.643581 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.639213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.5.122.130:46545)
0620 11:41:49.639607 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.639612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.639640 (+ 28us) server_negotiation.cc:407] Connection header received
0620 11:41:49.639836 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.639840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.639859 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.639945 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:49.640109 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.640933 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.642339 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.643121 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.643198 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.643296 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.643409 (+ 113us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.643450 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":74,"threads_started":1}
I20260620 11:41:49.644088 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.639374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:38069 (local address 127.0.0.1:57398)
0620 11:41:49.639781 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.639790 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.639841 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.640161 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.640162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.640164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:49.640195 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.640197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.641527 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.641528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.642550 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.642552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.643806 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.643807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.643873 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.643878 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.643933 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.643978 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":84,"threads_started":1}
I20260620 11:41:49.644345 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.639484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.5.122.129:38069)
0620 11:41:49.639659 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.639665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.640017 (+ 352us) server_negotiation.cc:407] Connection header received
0620 11:41:49.640075 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.640078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.640097 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.640168 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:49.640244 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.641428 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.642653 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.643430 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.643496 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.643601 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.644201 (+ 600us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.644259 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260620 11:41:49.644371 24461 tablet_service.cc:1467] Processing CreateTablet for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 (DEFAULT_TABLE table=test-table [id=5faa77396f164cceb8eb7cfb52c6a3ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:49.644704 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.639127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:32975 (local address 127.0.0.1:40008)
0620 11:41:49.639692 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.639710 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.639847 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.640196 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.640197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.640199 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:49.640244 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.640245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.642617 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.642617 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.643449 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.643451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.644466 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.644467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.644533 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.644538 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.644589 (+ 51us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.644631 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260620 11:41:49.644732 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.639383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.5.122.131:32975)
0620 11:41:49.639724 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.639728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.639766 (+ 38us) server_negotiation.cc:407] Connection header received
0620 11:41:49.639957 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.639960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.640046 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.640121 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:49.640308 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.641432 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.643543 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.644373 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.644466 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.644568 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.644611 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.644649 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":77,"threads_started":1}
I20260620 11:41:49.644770 24461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:49.646746 24330 tablet_service.cc:1467] Processing CreateTablet for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 (DEFAULT_TABLE table=test-table [id=5faa77396f164cceb8eb7cfb52c6a3ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:49.646799 24592 tablet_service.cc:1467] Processing CreateTablet for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 (DEFAULT_TABLE table=test-table [id=5faa77396f164cceb8eb7cfb52c6a3ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:49.647120 24330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:49.647151 24592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:49.648125 24672 tablet_bootstrap.cc:492] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Bootstrap starting.
I20260620 11:41:49.648960 24672 tablet_bootstrap.cc:654] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:49.649323 24672 log.cc:824] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:49.650249 24672 tablet_bootstrap.cc:492] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: No bootstrap required, opened a new log
I20260620 11:41:49.650332 24672 ts_tablet_manager.cc:1397] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:41:49.652546 24672 raft_consensus.cc:348] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.652709 24672 raft_consensus.cc:374] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:49.652734 24672 raft_consensus.cc:729] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e214c727844c42958d055518e97e19ea, State: Initialized, Role: FOLLOWER
I20260620 11:41:49.652856 24672 consensus_queue.cc:260] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.653139 24672 ts_tablet_manager.cc:1428] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:49.653162 24527 heartbeater.cc:503] Master 127.5.122.190:41149 was elected leader, sending a full tablet report...
I20260620 11:41:49.653343 24674 tablet_bootstrap.cc:492] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: Bootstrap starting.
I20260620 11:41:49.654080 24674 tablet_bootstrap.cc:654] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:49.654431 24674 log.cc:824] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:49.654850 24675 tablet_bootstrap.cc:492] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Bootstrap starting.
I20260620 11:41:49.655422 24674 tablet_bootstrap.cc:492] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: No bootstrap required, opened a new log
I20260620 11:41:49.655503 24674 ts_tablet_manager.cc:1397] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:41:49.655558 24675 tablet_bootstrap.cc:654] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:49.655881 24675 log.cc:824] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:49.656733 24675 tablet_bootstrap.cc:492] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: No bootstrap required, opened a new log
I20260620 11:41:49.656817 24675 ts_tablet_manager.cc:1397] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:41:49.658427 24674 raft_consensus.cc:348] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.658625 24674 raft_consensus.cc:374] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:49.658663 24674 raft_consensus.cc:729] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5975d6218a6040b4a9f5a4651096e255, State: Initialized, Role: FOLLOWER
I20260620 11:41:49.658911 24674 consensus_queue.cc:260] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.659227 24674 ts_tablet_manager.cc:1428] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260620 11:41:49.659374 24396 heartbeater.cc:503] Master 127.5.122.190:41149 was elected leader, sending a full tablet report...
I20260620 11:41:49.659601 24675 raft_consensus.cc:348] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.659807 24675 raft_consensus.cc:374] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:49.659847 24675 raft_consensus.cc:729] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b3b220718b4adb8bfa8e9fb80fa874, State: Initialized, Role: FOLLOWER
I20260620 11:41:49.659977 24675 consensus_queue.cc:260] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.660279 24675 ts_tablet_manager.cc:1428] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260620 11:41:49.660396 24658 heartbeater.cc:503] Master 127.5.122.190:41149 was elected leader, sending a full tablet report...
W20260620 11:41:49.721855 24528 tablet.cc:2369] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:41:49.834700 24397 tablet.cc:2369] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:49.841760 24679 raft_consensus.cc:482] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:49.841941 24679 raft_consensus.cc:504] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.842518 24679 leader_election.cc:288] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e214c727844c42958d055518e97e19ea (127.5.122.130:46545), 96b3b220718b4adb8bfa8e9fb80fa874 (127.5.122.131:32975)
I20260620 11:41:49.847085 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.842877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:48677 (local address 127.5.122.131:32975)
0620 11:41:49.842928 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.842931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.843010 (+ 79us) server_negotiation.cc:407] Connection header received
0620 11:41:49.843072 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.843073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.843084 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.843188 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:49.843489 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.844280 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.846003 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.846589 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.846683 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.846694 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.846959 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.847005 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260620 11:41:49.847085 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.842878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:40383 (local address 127.5.122.130:46545)
0620 11:41:49.842928 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.842931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.843009 (+ 78us) server_negotiation.cc:407] Connection header received
0620 11:41:49.843072 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.843073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.843086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.843165 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:49.843489 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.844265 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.846000 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.846594 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.846686 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.846694 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.846959 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.847005 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:41:49.847122 24681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.842772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:46545 (local address 127.5.122.129:40383)
0620 11:41:49.842920 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.842933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.843007 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.843237 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.843239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.843243 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:49.843383 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.843384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.844393 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.844395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.845845 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.845849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.846730 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.846732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.846857 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.846868 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.846939 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.847002 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":198,"thread_start_us":64,"threads_started":1}
I20260620 11:41:49.847126 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.842770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:32975 (local address 127.5.122.129:48677)
0620 11:41:49.842920 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.842933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:49.843007 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:49.843237 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:49.843239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:49.843242 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:49.843381 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.843383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.844393 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.844395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.845845 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:49.845849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.846733 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.846734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:49.846857 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.846867 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 11:41:49.846939 (+ 72us) client_negotiation.cc:240] Negotiation successful
0620 11:41:49.847002 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":99,"thread_start_us":76,"threads_started":1}
I20260620 11:41:49.847584 24481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c4c7982ed224b0887f2b6f5cc4fecb9" candidate_uuid: "5975d6218a6040b4a9f5a4651096e255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e214c727844c42958d055518e97e19ea" is_pre_election: true
I20260620 11:41:49.847787 24481 raft_consensus.cc:2454] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5975d6218a6040b4a9f5a4651096e255 in term 0.
I20260620 11:41:49.847829 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c4c7982ed224b0887f2b6f5cc4fecb9" candidate_uuid: "5975d6218a6040b4a9f5a4651096e255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" is_pre_election: true
I20260620 11:41:49.847985 24612 raft_consensus.cc:2454] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5975d6218a6040b4a9f5a4651096e255 in term 0.
I20260620 11:41:49.848115 24283 leader_election.cc:302] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5975d6218a6040b4a9f5a4651096e255, e214c727844c42958d055518e97e19ea; no voters:
I20260620 11:41:49.848264 24679 raft_consensus.cc:2781] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:41:49.848344 24679 raft_consensus.cc:482] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:41:49.848364 24679 raft_consensus.cc:3037] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:49.849260 24679 raft_consensus.cc:504] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.849502 24679 leader_election.cc:288] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [CANDIDATE]: Term 1 election: Requested vote from peers e214c727844c42958d055518e97e19ea (127.5.122.130:46545), 96b3b220718b4adb8bfa8e9fb80fa874 (127.5.122.131:32975)
I20260620 11:41:49.849603 24612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c4c7982ed224b0887f2b6f5cc4fecb9" candidate_uuid: "5975d6218a6040b4a9f5a4651096e255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b3b220718b4adb8bfa8e9fb80fa874"
I20260620 11:41:49.849668 24481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c4c7982ed224b0887f2b6f5cc4fecb9" candidate_uuid: "5975d6218a6040b4a9f5a4651096e255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e214c727844c42958d055518e97e19ea"
I20260620 11:41:49.849718 24612 raft_consensus.cc:3037] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:49.849783 24481 raft_consensus.cc:3037] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:49.850613 24481 raft_consensus.cc:2454] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5975d6218a6040b4a9f5a4651096e255 in term 1.
I20260620 11:41:49.850623 24612 raft_consensus.cc:2454] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5975d6218a6040b4a9f5a4651096e255 in term 1.
I20260620 11:41:49.850833 24283 leader_election.cc:302] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5975d6218a6040b4a9f5a4651096e255, e214c727844c42958d055518e97e19ea; no voters:
I20260620 11:41:49.850929 24679 raft_consensus.cc:2781] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:49.851122 24679 raft_consensus.cc:686] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 1 LEADER]: Becoming Leader. State: Replica: 5975d6218a6040b4a9f5a4651096e255, State: Running, Role: LEADER
I20260620 11:41:49.851397 24679 consensus_queue.cc:237] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } }
I20260620 11:41:49.852344 24208 catalog_manager.cc:5495] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5975d6218a6040b4a9f5a4651096e255 (127.5.122.129). New cstate: current_term: 1 leader_uuid: "5975d6218a6040b4a9f5a4651096e255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5975d6218a6040b4a9f5a4651096e255" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 38069 } health_report { overall_health: HEALTHY } } }
W20260620 11:41:49.859575 24659 tablet.cc:2369] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:49.875576 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.873244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.5.122.129:38069)
0620 11:41:49.873322 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:49.873324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:49.873334 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:41:49.873377 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:49.873378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:49.873399 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:49.873464 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:49.873592 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.874081 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.874818 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:49.875013 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:49.875069 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:49.875211 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:49.875306 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:41:49.875381 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:49.875436 (+ 55us) server_negotiation.cc:299] Negotiation successful
0620 11:41:49.875471 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 11:41:49.877681 24612 raft_consensus.cc:1261] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 1 FOLLOWER]: Refusing update from remote peer 5975d6218a6040b4a9f5a4651096e255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:49.877696 24481 raft_consensus.cc:1261] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 1 FOLLOWER]: Refusing update from remote peer 5975d6218a6040b4a9f5a4651096e255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:49.878113 24679 consensus_queue.cc:1035] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b3b220718b4adb8bfa8e9fb80fa874" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 32975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:49.878319 24683 consensus_queue.cc:1035] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e214c727844c42958d055518e97e19ea" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 46545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:49.881320 24688 mvcc.cc:205] Tried to move back new op lower bound from 7298890587655684096 to 7298890587552161792. Current Snapshot: MvccSnapshot[applied={T|T < 7298890587655684096}]
I20260620 11:41:49.884430 24208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:49.884527 24208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:49.885372 24208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 593a5355f5b2405f94115f05e09e7cd6: Sending DeleteTablet for 3 replicas of tablet 3c4c7982ed224b0887f2b6f5cc4fecb9
I20260620 11:41:49.885854 24461 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:49 UTC) from {username='slave'} at 127.0.0.1:55570
I20260620 11:41:49.885883 24592 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:49 UTC) from {username='slave'} at 127.0.0.1:40008
I20260620 11:41:49.885892 24330 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:49 UTC) from {username='slave'} at 127.0.0.1:57398
I20260620 11:41:49.886188 24698 tablet_replica.cc:330] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255: stopping tablet replica
I20260620 11:41:49.886299 24697 tablet_replica.cc:330] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: stopping tablet replica
I20260620 11:41:49.886374 24698 raft_consensus.cc:2229] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 5975d6218a6040b4a9f5a4651096e255 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:41:49.886456 24696 tablet_replica.cc:330] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: stopping tablet replica
I20260620 11:41:49.886543 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24268
I20260620 11:41:49.887655 24696 raft_consensus.cc:2229] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:49.887835 24697 raft_consensus.cc:2229] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:49.887969 24697 raft_consensus.cc:2258] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:49.888155 24696 raft_consensus.cc:2258] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:49.888619 24697 ts_tablet_manager.cc:1905] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:49.890327 24696 ts_tablet_manager.cc:1905] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:49.890359 24697 ts_tablet_manager.cc:1918] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:49.890427 24697 log.cc:1196] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/3c4c7982ed224b0887f2b6f5cc4fecb9
I20260620 11:41:49.890770 24697 ts_tablet_manager.cc:1939] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P 96b3b220718b4adb8bfa8e9fb80fa874: Deleting consensus metadata
I20260620 11:41:49.891254 24195 catalog_manager.cc:4841] TS 96b3b220718b4adb8bfa8e9fb80fa874 (127.5.122.131:32975): tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 (table test-table [id=5faa77396f164cceb8eb7cfb52c6a3ba]) successfully deleted
I20260620 11:41:49.892354 24696 ts_tablet_manager.cc:1918] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:49.892417 24696 log.cc:1196] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/3c4c7982ed224b0887f2b6f5cc4fecb9
I20260620 11:41:49.892695 24696 ts_tablet_manager.cc:1939] T 3c4c7982ed224b0887f2b6f5cc4fecb9 P e214c727844c42958d055518e97e19ea: Deleting consensus metadata
I20260620 11:41:49.893245 24192 catalog_manager.cc:4841] TS e214c727844c42958d055518e97e19ea (127.5.122.130:46545): tablet 3c4c7982ed224b0887f2b6f5cc4fecb9 (table test-table [id=5faa77396f164cceb8eb7cfb52c6a3ba]) successfully deleted
W20260620 11:41:49.893258 24195 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:38069 (error 108)
I20260620 11:41:49.893584 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24399
I20260620 11:41:49.893937 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:49.893789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:38069 (local address 127.0.0.1:57422)
0620 11:41:49.893851 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:49.893894 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260620 11:41:49.894299 24195 catalog_manager.cc:4584] TS 5975d6218a6040b4a9f5a4651096e255 (127.5.122.129:38069): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c4c7982ed224b0887f2b6f5cc4fecb9: Network error: Client connection negotiation failed: client connection to 127.5.122.129:38069: connect: Connection refused (error 111)
I20260620 11:41:49.899708 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24530
I20260620 11:41:49.905316 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24177
2026-06-20T11:41:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (884 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:49 dist-test-slave-szdr krb5kdc[24702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:49 dist-test-slave-szdr krb5kdc[24702](info): set up 2 sockets
Jun 20 11:41:49 dist-test-slave-szdr krb5kdc[24702](info): commencing operation
krb5kdc: starting...
W20260620 11:41:50.947177 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:41:51.076565 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:42893
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:42893
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:51.160306 24715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:51.160642 24715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:51.160688 24715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:51.162042 24715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:51.162137 24715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:51.162160 24715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:51.162184 24715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:51.162249 24715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:51.163914 24715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:42893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:42893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:51.164251 24715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:51.164508 24715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:51.170003 24723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.170022 24721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.170025 24720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:51.170149 24715 server_base.cc:1034] running on GCE node
I20260620 11:41:51.170614 24715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:51.170979 24715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:51.172179 24715 hybrid_clock.cc:648] HybridClock initialized: now 1781955711172144 us; error 51 us; skew 500 ppm
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:51.176352 24715 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:41:51.177083 24715 webserver.cc:460] Webserver started at http://127.5.122.190:41775/ using document root <none> and password file <none>
I20260620 11:41:51.177413 24715 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:51.177502 24715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:51.177673 24715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:51.179319 24715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "8c5380c1ecf44e0abc3fe8458d1afe87"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "5105a880208c80329bc6c81b467de0e3"
server_key_iv: "51cfe8964ff04b5df91b8f7beb2c2b0e"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.179828 24715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "8c5380c1ecf44e0abc3fe8458d1afe87"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "5105a880208c80329bc6c81b467de0e3"
server_key_iv: "51cfe8964ff04b5df91b8f7beb2c2b0e"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.181589 24715 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 11:41:51.182622 24730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.182835 24715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:41:51.182967 24715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "8c5380c1ecf44e0abc3fe8458d1afe87"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "5105a880208c80329bc6c81b467de0e3"
server_key_iv: "51cfe8964ff04b5df91b8f7beb2c2b0e"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.183090 24715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:51.209579 24715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:51.212648 24715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:51.212853 24715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:51.217624 24715 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:42893
I20260620 11:41:51.217659 24782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:42893 every 8 connection(s)
I20260620 11:41:51.218180 24715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:51.219270 24783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:51.221839 24783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87: Bootstrap starting.
I20260620 11:41:51.222151 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24715
I20260620 11:41:51.222291 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:51.222520 5610 external_mini_cluster.cc:1428] Setting key 7b2f82aa0aa6aa18b1ece2316c57cac9
I20260620 11:41:51.222786 24783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:51.223145 24783 log.cc:824] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:51.223893 24783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87: No bootstrap required, opened a new log
I20260620 11:41:51.225852 24783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42893 } }
I20260620 11:41:51.226095 24783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:51.226125 24783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c5380c1ecf44e0abc3fe8458d1afe87, State: Initialized, Role: FOLLOWER
I20260620 11:41:51.226316 24783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42893 } }
I20260620 11:41:51.226421 24783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:51.226467 24783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:51.226518 24783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:51.227167 24783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42893 } }
I20260620 11:41:51.227312 24783 leader_election.cc:302] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c5380c1ecf44e0abc3fe8458d1afe87; no voters:
I20260620 11:41:51.227564 24783 leader_election.cc:288] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:51.227595 24788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:51.227728 24788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [term 1 LEADER]: Becoming Leader. State: Replica: 8c5380c1ecf44e0abc3fe8458d1afe87, State: Running, Role: LEADER
I20260620 11:41:51.227815 24788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42893 } }
I20260620 11:41:51.227854 24783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:51.228153 24788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c5380c1ecf44e0abc3fe8458d1afe87. Latest consensus state: current_term: 1 leader_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42893 } } }
I20260620 11:41:51.228228 24788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:51.228173 24790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c5380c1ecf44e0abc3fe8458d1afe87" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 42893 } } }
I20260620 11:41:51.228456 24790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87 [sys.catalog]: This master's current role is: LEADER
W20260620 11:41:51.229732 24803 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:41:51.230330 24803 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:41:51.230417 24793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:51.230618 24793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:51.232863 24793 catalog_manager.cc:1347] Generated new cluster ID: 341fd6721c2a4cf79bef5b538c4f1268
I20260620 11:41:51.233018 24793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:51.235257 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.223139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.5.122.190:42893)
0620 11:41:51.223431 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.223440 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.223469 (+ 29us) server_negotiation.cc:407] Connection header received
0620 11:41:51.224074 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.224083 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.224141 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.224315 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:51.225046 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.226263 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.227251 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.228053 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.228838 (+ 785us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.231443 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:51.231448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:51.231448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:51.231462 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:51.233476 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.233916 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.233920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.233920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.234061 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.234290 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.234292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.234292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.234564 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:51.234742 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.234858 (+ 116us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.235013 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":87,"threads_started":1}
I20260620 11:41:51.241147 24793 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:51.242061 24793 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:51.247262 24793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87: Generated new TSK 0
I20260620 11:41:51.247500 24793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.122.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
Entry for principal kudu/127.5.122.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab.
I20260620 11:41:51.281414 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42893
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:51.368551 24809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:51.368799 24809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:51.368829 24809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:51.370175 24809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:51.370224 24809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:51.370256 24809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:51.371845 24809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.129.keytab
--principal=kudu/127.5.122.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:51.372053 24809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:51.372249 24809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:51.377592 24815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.377666 24814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.377722 24817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:51.378037 24809 server_base.cc:1034] running on GCE node
I20260620 11:41:51.378285 24809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:51.378571 24809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:51.379752 24809 hybrid_clock.cc:648] HybridClock initialized: now 1781955711379740 us; error 33 us; skew 500 ppm
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:51.382881 24809 init.cc:377] Logged in from keytab as kudu/127.5.122.129@KRBTEST.COM (short username kudu)
I20260620 11:41:51.383399 24809 webserver.cc:460] Webserver started at http://127.5.122.129:35101/ using document root <none> and password file <none>
I20260620 11:41:51.383648 24809 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:51.383723 24809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:51.383876 24809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:51.385089 24809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "7865ef1b12a040eaace85957a4ccf21b"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "8f89064c97bc0539fedba1cc20735efe"
server_key_iv: "7a5a96f9e656460158984a4451d316af"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.385499 24809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "7865ef1b12a040eaace85957a4ccf21b"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "8f89064c97bc0539fedba1cc20735efe"
server_key_iv: "7a5a96f9e656460158984a4451d316af"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.387298 24809 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260620 11:41:51.388236 24824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.388553 24809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:51.388674 24809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "7865ef1b12a040eaace85957a4ccf21b"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "8f89064c97bc0539fedba1cc20735efe"
server_key_iv: "7a5a96f9e656460158984a4451d316af"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.388775 24809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:51.403110 24809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:51.405933 24809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:51.406276 24809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:51.406554 24809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:51.406917 24809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:51.406984 24809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.407022 24809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:51.407043 24809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.413491 24809 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:41217
I20260620 11:41:51.413578 24937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:41217 every 8 connection(s)
I20260620 11:41:51.413954 24809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:41:51.417349 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24809
I20260620 11:41:51.417474 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:41:51.417708 5610 external_mini_cluster.cc:1428] Setting key a5a32c66bd962f13d4f18be60a5974d4
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.129@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:51.425639 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.414828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:59951 (local address 127.5.122.190:42893)
0620 11:41:51.414924 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.414927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.415106 (+ 179us) server_negotiation.cc:407] Connection header received
0620 11:41:51.415700 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.415702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.415714 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.415777 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:51.416813 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.417586 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.419111 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.419480 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.419566 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.422405 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:51.422407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:51.422407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:51.422416 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:51.424025 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.424548 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.424550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.424550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.424620 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.424906 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.424908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.424908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.425041 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:51.425122 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.425436 (+ 314us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.425536 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 11:41:51.425877 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.414773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:42893 (local address 127.5.122.129:59951)
0620 11:41:51.414963 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.415005 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.415567 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.415844 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.415848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.416472 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:51.416691 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.416696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.417701 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.417702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.418915 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.418919 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.419586 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.419587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.419717 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.420405 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:51.420418 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:51.422288 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:51.424160 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:51.424171 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:51.424174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:51.424462 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:51.424699 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:51.424700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:51.424700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:51.424838 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:51.425114 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:51.425122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:51.425288 (+ 166us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.425402 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.425604 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260620 11:41:51.426533 24938 heartbeater.cc:347] Connected to a master server at 127.5.122.190:42893
WARNING: no policy specified for kudu/127.5.122.130@KRBTEST.COM; defaulting to no policy
I20260620 11:41:51.426674 24938 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:51.426951 24938 heartbeater.cc:511] Master 127.5.122.190:42893 requested a full tablet report, sending...
I20260620 11:41:51.427603 24747 ts_manager.cc:194] Registered new tserver with Master: 7865ef1b12a040eaace85957a4ccf21b (127.5.122.129:41217)
I20260620 11:41:51.428507 24747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.129@KRBTEST.COM'} at 127.5.122.129:59951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
Entry for principal kudu/127.5.122.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab.
I20260620 11:41:51.447315 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42893
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:51.533015 24943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:51.533259 24943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:51.533288 24943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:51.534530 24943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:51.534571 24943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:51.534597 24943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:51.536072 24943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.130.keytab
--principal=kudu/127.5.122.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:51.536295 24943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:51.536481 24943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:51.541920 24948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.541986 24951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.541934 24949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:51.542208 24943 server_base.cc:1034] running on GCE node
I20260620 11:41:51.542506 24943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:51.542781 24943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:51.543955 24943 hybrid_clock.cc:648] HybridClock initialized: now 1781955711543949 us; error 39 us; skew 500 ppm
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:51.546718 24943 init.cc:377] Logged in from keytab as kudu/127.5.122.130@KRBTEST.COM (short username kudu)
I20260620 11:41:51.547245 24943 webserver.cc:460] Webserver started at http://127.5.122.130:38617/ using document root <none> and password file <none>
I20260620 11:41:51.547483 24943 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:51.547551 24943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:51.547714 24943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:51.548839 24943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "e51cda6d4b8e4ffdb672861375a9ba6a"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "989f54c9b352fbf36adf4503416903fb"
server_key_iv: "331018bc5582f7bd131f08c838c1e401"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.549230 24943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "e51cda6d4b8e4ffdb672861375a9ba6a"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "989f54c9b352fbf36adf4503416903fb"
server_key_iv: "331018bc5582f7bd131f08c838c1e401"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.550750 24943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:51.551630 24958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.551836 24943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:51.551942 24943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "e51cda6d4b8e4ffdb672861375a9ba6a"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "989f54c9b352fbf36adf4503416903fb"
server_key_iv: "331018bc5582f7bd131f08c838c1e401"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.552021 24943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:51.569293 24943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:51.571650 24943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:51.571846 24943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:51.572124 24943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:51.572468 24943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:51.572526 24943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.572562 24943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:51.572604 24943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.579048 24943 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:34927
I20260620 11:41:51.579090 25071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:34927 every 8 connection(s)
I20260620 11:41:51.579452 24943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:51.583191 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 24943
I20260620 11:41:51.583356 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:41:51.583555 5610 external_mini_cluster.cc:1428] Setting key b2b57ee39978d1d940f56f296b4329d1
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.130@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:51.590456 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.580424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:34921 (local address 127.5.122.190:42893)
0620 11:41:51.580523 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.580525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.580710 (+ 185us) server_negotiation.cc:407] Connection header received
0620 11:41:51.581290 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.581292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.581305 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.581356 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:51.582495 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.583013 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.584388 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.584649 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.584707 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.587417 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:51.587418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:51.587419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:51.587427 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:51.588904 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.589349 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.589350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.589351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.589414 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.589684 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.589685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.589686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.589825 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:51.589898 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.590257 (+ 359us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.590358 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 11:41:51.590675 25074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.580319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:42893 (local address 127.5.122.130:34921)
0620 11:41:51.580557 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.580615 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.581170 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.581432 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.581436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.582153 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:51.582386 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.582392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.583122 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.583123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.584167 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.584170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.584740 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.584741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.584892 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.585507 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:51.585518 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:51.587305 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:51.588994 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:51.589003 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:51.589006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:51.589248 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:51.589488 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:51.589489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:51.589489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:51.589607 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:51.589901 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:51.589909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:51.590120 (+ 211us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.590237 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.590423 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":121,"threads_started":1}
I20260620 11:41:51.591180 25072 heartbeater.cc:347] Connected to a master server at 127.5.122.190:42893
I20260620 11:41:51.591318 25072 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:51.591521 25072 heartbeater.cc:511] Master 127.5.122.190:42893 requested a full tablet report, sending...
I20260620 11:41:51.591993 24747 ts_manager.cc:194] Registered new tserver with Master: e51cda6d4b8e4ffdb672861375a9ba6a (127.5.122.130:34927)
I20260620 11:41:51.592404 24747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.130@KRBTEST.COM'} at 127.5.122.130:34921
WARNING: no policy specified for kudu/127.5.122.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
Entry for principal kudu/127.5.122.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab.
I20260620 11:41:51.614456 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42893
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:34085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:51.699928 25077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:51.700189 25077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:51.700238 25077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:51.701529 25077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:51.701613 25077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:51.701659 25077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:51.703377 25077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.131.keytab
--principal=kudu/127.5.122.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:42893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:51.703686 25077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:51.703951 25077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:51.709458 25085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.709494 25082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:51.709477 25083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:51.709900 25077 server_base.cc:1034] running on GCE node
I20260620 11:41:51.710175 25077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:51.710428 25077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:51.711588 25077 hybrid_clock.cc:648] HybridClock initialized: now 1781955711711556 us; error 52 us; skew 500 ppm
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:51.714354 25077 init.cc:377] Logged in from keytab as kudu/127.5.122.131@KRBTEST.COM (short username kudu)
I20260620 11:41:51.714886 25077 webserver.cc:460] Webserver started at http://127.5.122.131:33997/ using document root <none> and password file <none>
I20260620 11:41:51.715147 25077 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:51.715225 25077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:51.715379 25077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:51.716552 25077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "4dfbeee9aec34ea48f5542e4579169da"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "f21167606d7b0fb7017791b93812fdc6"
server_key_iv: "a35c8cfe9f3546a6192ed6055168e57b"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.716925 25077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "4dfbeee9aec34ea48f5542e4579169da"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "f21167606d7b0fb7017791b93812fdc6"
server_key_iv: "a35c8cfe9f3546a6192ed6055168e57b"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.718430 25077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:51.719537 25092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.719884 25077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:51.720026 25077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "4dfbeee9aec34ea48f5542e4579169da"
format_stamp: "Formatted at 2026-06-20 11:41:51 on dist-test-slave-szdr"
server_key: "f21167606d7b0fb7017791b93812fdc6"
server_key_iv: "a35c8cfe9f3546a6192ed6055168e57b"
server_key_version: "encryptionkey@0"
I20260620 11:41:51.720115 25077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:51.738760 25077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:51.741557 25077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:51.741792 25077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:51.742141 25077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:51.742502 25077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:51.742563 25077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.742607 25077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:51.742635 25077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:51.749161 25077 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:42221
I20260620 11:41:51.749200 25205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:42221 every 8 connection(s)
I20260620 11:41:51.749559 25077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:51.750501 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 25077
I20260620 11:41:51.750644 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:41:51.750835 5610 external_mini_cluster.cc:1428] Setting key d83b4d4a4751259d2b5dbb931238d7ec
Jun 20 11:41:51 dist-test-slave-szdr krb5kdc[24702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955711, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.131@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:41:51.760519 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.750459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:43869 (local address 127.5.122.190:42893)
0620 11:41:51.750540 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.750543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.750935 (+ 392us) server_negotiation.cc:407] Connection header received
0620 11:41:51.751590 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.751591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.751604 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.751676 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:51.752788 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.753360 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.754661 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.754897 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.755038 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.757534 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:51.757536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:51.757536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:51.757543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:51.759109 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.759536 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.759537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.759537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.759593 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.759825 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.759825 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.759826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.759949 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:51.760004 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.760369 (+ 365us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.760446 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260620 11:41:51.760716 25208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.750440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:42893 (local address 127.5.122.131:43869)
0620 11:41:51.750773 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.750822 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.751475 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.751766 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.751771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.752441 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:51.752677 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.752682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.753480 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.753482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.754532 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.754535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.755066 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.755067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.755224 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.755765 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:41:51.755776 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:51.757421 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:51.759199 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:51.759208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:51.759211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:51.759463 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:51.759655 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:41:51.759656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:41:51.759656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:41:51.759767 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:41:51.760039 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:51.760046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:51.760219 (+ 173us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.760325 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.760472 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":208,"threads_started":1}
I20260620 11:41:51.761230 25206 heartbeater.cc:347] Connected to a master server at 127.5.122.190:42893
I20260620 11:41:51.761332 25206 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:51.761497 25206 heartbeater.cc:511] Master 127.5.122.190:42893 requested a full tablet report, sending...
I20260620 11:41:51.761904 24747 ts_manager.cc:194] Registered new tserver with Master: 4dfbeee9aec34ea48f5542e4579169da (127.5.122.131:42221)
I20260620 11:41:51.762480 24747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.122.131@KRBTEST.COM'} at 127.5.122.131:43869
I20260620 11:41:51.762779 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:41:51.771106 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.764285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.5.122.190:42893)
0620 11:41:51.764345 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.764347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.764480 (+ 133us) server_negotiation.cc:407] Connection header received
0620 11:41:51.764560 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.764562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.764572 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.764629 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:51.765235 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.765732 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.766582 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.766797 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.766879 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.768335 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:51.768338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:51.768338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:51.768346 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:51.769789 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.770229 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.770230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.770230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.770293 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:51.770509 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:51.770509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:51.770510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:51.770678 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:51.770795 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.770902 (+ 107us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.770991 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:41:51.773195 24747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:41:51.775216 24747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:41:51.783083 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.778542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.5.122.131:42221)
0620 11:41:51.778789 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.778794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.778805 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:41:51.778851 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.778853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.778869 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.778953 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:51.779150 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.780325 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.781652 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.782529 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.782619 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.782706 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.782915 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.782971 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260620 11:41:51.783093 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.778327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.5.122.129:41217)
0620 11:41:51.778558 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.778563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.778747 (+ 184us) server_negotiation.cc:407] Connection header received
0620 11:41:51.778801 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.778804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.778823 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.778989 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:51.779379 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.780091 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.781815 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.782603 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.782665 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.782715 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.782943 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.782988 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":100,"threads_started":1}
I20260620 11:41:51.783097 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.778182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:34927 (local address 127.0.0.1:50460)
0620 11:41:51.778395 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.778555 (+ 160us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.778640 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.778966 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.778969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.778971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:51.779055 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.779057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.780462 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.780463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.781712 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.781715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.782755 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.782755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.782834 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.782840 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.782890 (+ 50us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.782995 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":8,"thread_start_us":82,"threads_started":1}
I20260620 11:41:51.783099 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.778434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:42221 (local address 127.0.0.1:32860)
0620 11:41:51.778735 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.778745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.778786 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.779024 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.779025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.779026 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:51.779064 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.779065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.780466 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.780467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.781543 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.781546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.782634 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.782635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.782765 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.782787 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.782893 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.782997 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":34,"thread_start_us":71,"threads_started":1}
I20260620 11:41:51.783377 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.778206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:41217 (local address 127.0.0.1:58638)
0620 11:41:51.778555 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.778567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.778642 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.779242 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.779243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.779245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:51.779298 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.779300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.780212 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.780214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.781717 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.781719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.782752 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.782753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.782834 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.782838 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.782890 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.783289 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":430,"thread_start_us":92,"threads_started":1}
I20260620 11:41:51.783568 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.778287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50460 (local address 127.5.122.130:34927)
0620 11:41:51.778662 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.778666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.778677 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:41:51.778767 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.778770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.778789 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.778864 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:51.779142 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.780363 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.782063 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.782624 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.782700 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.782760 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.783411 (+ 651us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.783475 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":81,"threads_started":1}
I20260620 11:41:51.784255 24872 tablet_service.cc:1467] Processing CreateTablet for tablet 538010cbc7d44014bd7ba8d5adf7f787 (DEFAULT_TABLE table=test-table [id=7bac476a048a429283424b7f28d702c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:51.784255 25006 tablet_service.cc:1467] Processing CreateTablet for tablet 538010cbc7d44014bd7ba8d5adf7f787 (DEFAULT_TABLE table=test-table [id=7bac476a048a429283424b7f28d702c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:51.784427 25140 tablet_service.cc:1467] Processing CreateTablet for tablet 538010cbc7d44014bd7ba8d5adf7f787 (DEFAULT_TABLE table=test-table [id=7bac476a048a429283424b7f28d702c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:51.784658 24872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538010cbc7d44014bd7ba8d5adf7f787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:51.784668 25006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538010cbc7d44014bd7ba8d5adf7f787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:51.784817 25140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 538010cbc7d44014bd7ba8d5adf7f787. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:51.788090 25223 tablet_bootstrap.cc:492] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Bootstrap starting.
I20260620 11:41:51.788594 25224 tablet_bootstrap.cc:492] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Bootstrap starting.
I20260620 11:41:51.788594 25225 tablet_bootstrap.cc:492] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b: Bootstrap starting.
I20260620 11:41:51.788944 25223 tablet_bootstrap.cc:654] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:51.789284 25225 tablet_bootstrap.cc:654] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:51.789283 25224 tablet_bootstrap.cc:654] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:51.789518 25223 log.cc:824] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:51.789885 25225 log.cc:824] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:51.789917 25224 log.cc:824] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:51.790720 25225 tablet_bootstrap.cc:492] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b: No bootstrap required, opened a new log
I20260620 11:41:51.790743 25223 tablet_bootstrap.cc:492] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: No bootstrap required, opened a new log
I20260620 11:41:51.790809 25223 ts_tablet_manager.cc:1397] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260620 11:41:51.790812 25225 ts_tablet_manager.cc:1397] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260620 11:41:51.790923 25224 tablet_bootstrap.cc:492] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: No bootstrap required, opened a new log
I20260620 11:41:51.790984 25224 ts_tablet_manager.cc:1397] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:41:51.792433 25225 raft_consensus.cc:348] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.792614 25225 raft_consensus.cc:374] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:51.792658 25225 raft_consensus.cc:729] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7865ef1b12a040eaace85957a4ccf21b, State: Initialized, Role: FOLLOWER
I20260620 11:41:51.792788 25225 consensus_queue.cc:260] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.792917 25223 raft_consensus.cc:348] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.793095 25223 raft_consensus.cc:374] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:51.793118 24938 heartbeater.cc:503] Master 127.5.122.190:42893 was elected leader, sending a full tablet report...
I20260620 11:41:51.793133 25223 raft_consensus.cc:729] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dfbeee9aec34ea48f5542e4579169da, State: Initialized, Role: FOLLOWER
I20260620 11:41:51.793080 25224 raft_consensus.cc:348] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.793255 25223 consensus_queue.cc:260] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.793300 25224 raft_consensus.cc:374] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:51.793349 25224 raft_consensus.cc:729] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51cda6d4b8e4ffdb672861375a9ba6a, State: Initialized, Role: FOLLOWER
I20260620 11:41:51.793105 25225 ts_tablet_manager.cc:1428] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 11:41:51.793480 25224 consensus_queue.cc:260] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.793561 25223 ts_tablet_manager.cc:1428] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:51.793694 25206 heartbeater.cc:503] Master 127.5.122.190:42893 was elected leader, sending a full tablet report...
I20260620 11:41:51.793763 25224 ts_tablet_manager.cc:1428] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:51.793891 25072 heartbeater.cc:503] Master 127.5.122.190:42893 was elected leader, sending a full tablet report...
W20260620 11:41:51.830060 25073 tablet.cc:2369] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:41:51.914846 24939 tablet.cc:2369] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:51.938746 25231 raft_consensus.cc:482] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:51.938884 25231 raft_consensus.cc:504] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.939216 25231 leader_election.cc:288] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dfbeee9aec34ea48f5542e4579169da (127.5.122.131:42221), 7865ef1b12a040eaace85957a4ccf21b (127.5.122.129:41217)
I20260620 11:41:51.942473 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.939553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:44915 (local address 127.5.122.129:41217)
0620 11:41:51.939604 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.939607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.939802 (+ 195us) server_negotiation.cc:407] Connection header received
0620 11:41:51.939857 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.939858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.939870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.939943 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:51.940130 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.940630 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.941631 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.942093 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.942184 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.942194 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.942347 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.942400 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:41:51.942483 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.939496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:41217 (local address 127.5.122.130:44915)
0620 11:41:51.939713 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.939725 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.939786 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.939981 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.939982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.939983 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:51.940056 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.940057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.940733 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.940734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.941503 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.941505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.942183 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.942184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.942271 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.942275 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.942322 (+ 47us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.942359 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":18,"thread_start_us":133,"threads_started":1}
I20260620 11:41:51.942561 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.939582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:49135 (local address 127.5.122.131:42221)
0620 11:41:51.939626 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.939628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.939647 (+ 19us) server_negotiation.cc:407] Connection header received
0620 11:41:51.939688 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.939689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.939699 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.939788 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:51.939981 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.940551 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.941581 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.942093 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.942183 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.942193 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.942433 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.942482 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:41:51.942585 25074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.939494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:42221 (local address 127.5.122.130:49135)
0620 11:41:51.939569 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.939583 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:51.939633 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:51.939815 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:51.939816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:51.939819 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:51.939902 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.939904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.940647 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.940648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.941472 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:51.941475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.942241 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.942246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:51.942339 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.942343 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:51.942403 (+ 60us) client_negotiation.cc:240] Negotiation successful
0620 11:41:51.942450 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 11:41:51.943104 24892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538010cbc7d44014bd7ba8d5adf7f787" candidate_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7865ef1b12a040eaace85957a4ccf21b" is_pre_election: true
I20260620 11:41:51.943142 25160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538010cbc7d44014bd7ba8d5adf7f787" candidate_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dfbeee9aec34ea48f5542e4579169da" is_pre_election: true
I20260620 11:41:51.943284 24892 raft_consensus.cc:2454] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51cda6d4b8e4ffdb672861375a9ba6a in term 0.
I20260620 11:41:51.943310 25160 raft_consensus.cc:2454] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e51cda6d4b8e4ffdb672861375a9ba6a in term 0.
I20260620 11:41:51.943557 24960 leader_election.cc:302] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7865ef1b12a040eaace85957a4ccf21b, e51cda6d4b8e4ffdb672861375a9ba6a; no voters:
I20260620 11:41:51.943771 25231 raft_consensus.cc:2781] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:41:51.943845 25231 raft_consensus.cc:482] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:41:51.943882 25231 raft_consensus.cc:3037] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:51.944618 25231 raft_consensus.cc:504] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.944787 25231 leader_election.cc:288] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [CANDIDATE]: Term 1 election: Requested vote from peers 4dfbeee9aec34ea48f5542e4579169da (127.5.122.131:42221), 7865ef1b12a040eaace85957a4ccf21b (127.5.122.129:41217)
I20260620 11:41:51.944957 25160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538010cbc7d44014bd7ba8d5adf7f787" candidate_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dfbeee9aec34ea48f5542e4579169da"
I20260620 11:41:51.944972 24892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "538010cbc7d44014bd7ba8d5adf7f787" candidate_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7865ef1b12a040eaace85957a4ccf21b"
I20260620 11:41:51.945044 25160 raft_consensus.cc:3037] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:51.945068 24892 raft_consensus.cc:3037] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:51.945814 25160 raft_consensus.cc:2454] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51cda6d4b8e4ffdb672861375a9ba6a in term 1.
I20260620 11:41:51.945822 24892 raft_consensus.cc:2454] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e51cda6d4b8e4ffdb672861375a9ba6a in term 1.
I20260620 11:41:51.946025 24961 leader_election.cc:302] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dfbeee9aec34ea48f5542e4579169da, e51cda6d4b8e4ffdb672861375a9ba6a; no voters:
I20260620 11:41:51.946132 25231 raft_consensus.cc:2781] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:51.946328 25231 raft_consensus.cc:686] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 1 LEADER]: Becoming Leader. State: Replica: e51cda6d4b8e4ffdb672861375a9ba6a, State: Running, Role: LEADER
I20260620 11:41:51.946465 25231 consensus_queue.cc:237] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } }
I20260620 11:41:51.948100 24745 catalog_manager.cc:5495] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a reported cstate change: term changed from 0 to 1, leader changed from <none> to e51cda6d4b8e4ffdb672861375a9ba6a (127.5.122.130). New cstate: current_term: 1 leader_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e51cda6d4b8e4ffdb672861375a9ba6a" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 } health_report { overall_health: UNKNOWN } } }
I20260620 11:41:51.966717 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.964045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.5.122.130:34927)
0620 11:41:51.964126 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:51.964128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:51.964137 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:41:51.964181 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:51.964182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:51.964202 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:51.964326 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:51.964494 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.965045 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.965827 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:51.966086 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:51.966171 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:51.966352 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:51.966475 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:41:51.966573 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:51.966591 (+ 18us) server_negotiation.cc:299] Negotiation successful
0620 11:41:51.966613 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 11:41:51.968910 24892 raft_consensus.cc:1261] T 538010cbc7d44014bd7ba8d5adf7f787 P 7865ef1b12a040eaace85957a4ccf21b [term 1 FOLLOWER]: Refusing update from remote peer e51cda6d4b8e4ffdb672861375a9ba6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:51.968972 25160 raft_consensus.cc:1261] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 1 FOLLOWER]: Refusing update from remote peer e51cda6d4b8e4ffdb672861375a9ba6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:51.969290 25231 consensus_queue.cc:1035] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7865ef1b12a040eaace85957a4ccf21b" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 41217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:51.969527 25233 consensus_queue.cc:1035] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dfbeee9aec34ea48f5542e4579169da" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 42221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:51.972700 25236 mvcc.cc:205] Tried to move back new op lower bound from 7298890596221415424 to 7298890596133601280. Current Snapshot: MvccSnapshot[applied={T|T < 7298890596221415424 or (T in {7298890596221415424})}]
I20260620 11:41:51.974030 25238 mvcc.cc:205] Tried to move back new op lower bound from 7298890596221415424 to 7298890596133601280. Current Snapshot: MvccSnapshot[applied={T|T < 7298890596221415424 or (T in {7298890596221415424})}]
I20260620 11:41:51.974548 24747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:51.974668 24747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:51.975477 24747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c5380c1ecf44e0abc3fe8458d1afe87: Sending DeleteTablet for 3 replicas of tablet 538010cbc7d44014bd7ba8d5adf7f787
I20260620 11:41:51.975811 25006 tablet_service.cc:1514] Processing DeleteTablet for tablet 538010cbc7d44014bd7ba8d5adf7f787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:51 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:50460
I20260620 11:41:51.975859 24872 tablet_service.cc:1514] Processing DeleteTablet for tablet 538010cbc7d44014bd7ba8d5adf7f787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:51 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:58638
I20260620 11:41:51.976080 25245 tablet_replica.cc:330] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: stopping tablet replica
I20260620 11:41:51.976107 25140 tablet_service.cc:1514] Processing DeleteTablet for tablet 538010cbc7d44014bd7ba8d5adf7f787 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:51 UTC) from {username='kudu', principal='kudu/127.5.122.190@KRBTEST.COM'} at 127.0.0.1:32860
I20260620 11:41:51.976190 25245 raft_consensus.cc:2229] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:41:51.976289 25245 raft_consensus.cc:2258] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:51.976368 25247 tablet_replica.cc:330] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: stopping tablet replica
I20260620 11:41:51.976470 25247 raft_consensus.cc:2229] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:51.976585 25247 raft_consensus.cc:2258] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:51.976816 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24809
I20260620 11:41:51.977041 25247 ts_tablet_manager.cc:1905] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:51.979162 25247 ts_tablet_manager.cc:1918] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:51.979276 25247 log.cc:1196] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/538010cbc7d44014bd7ba8d5adf7f787
I20260620 11:41:51.979559 25247 ts_tablet_manager.cc:1939] T 538010cbc7d44014bd7ba8d5adf7f787 P 4dfbeee9aec34ea48f5542e4579169da: Deleting consensus metadata
I20260620 11:41:51.980019 24733 catalog_manager.cc:4841] TS 4dfbeee9aec34ea48f5542e4579169da (127.5.122.131:42221): tablet 538010cbc7d44014bd7ba8d5adf7f787 (table test-table [id=7bac476a048a429283424b7f28d702c8]) successfully deleted
I20260620 11:41:51.980835 25245 ts_tablet_manager.cc:1905] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:51.982365 25245 ts_tablet_manager.cc:1918] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:51.982470 25245 log.cc:1196] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/538010cbc7d44014bd7ba8d5adf7f787
I20260620 11:41:51.982734 25245 ts_tablet_manager.cc:1939] T 538010cbc7d44014bd7ba8d5adf7f787 P e51cda6d4b8e4ffdb672861375a9ba6a: Deleting consensus metadata
I20260620 11:41:51.983156 24733 catalog_manager.cc:4841] TS e51cda6d4b8e4ffdb672861375a9ba6a (127.5.122.130:34927): tablet 538010cbc7d44014bd7ba8d5adf7f787 (table test-table [id=7bac476a048a429283424b7f28d702c8]) successfully deleted
W20260620 11:41:51.984849 24732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:41217 (error 108)
I20260620 11:41:51.985390 25217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:51.985256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:41217 (local address 127.0.0.1:58644)
0620 11:41:51.985317 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:51.985361 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:41217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 11:41:51.985460 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24943
W20260620 11:41:51.985697 24732 catalog_manager.cc:4584] TS 7865ef1b12a040eaace85957a4ccf21b (127.5.122.129:41217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 538010cbc7d44014bd7ba8d5adf7f787: Network error: Client connection negotiation failed: client connection to 127.5.122.129:41217: connect: Connection refused (error 111)
I20260620 11:41:51.991739 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 25077
I20260620 11:41:51.996497 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 24715
2026-06-20T11:41:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2091 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T11:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:52Z Disabled control of system clock
I20260620 11:41:52.026245 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36619
--webserver_interface=127.5.122.190
--webserver_port=0
--builtin_ntp_servers=127.5.122.148:34017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:36619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:52.110008 25252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:52.110317 25252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:52.110357 25252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:52.111647 25252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:52.111745 25252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:52.111768 25252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:52.111789 25252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:52.111821 25252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:52.113301 25252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:36619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:52.113631 25252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:52.114009 25252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:41:52.119592 25252 server_base.cc:1034] running on GCE node
W20260620 11:41:52.119534 25258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:52.119534 25257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:52.119556 25260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:52.120106 25252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:52.120447 25252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:52.121601 25252 hybrid_clock.cc:648] HybridClock initialized: now 1781955712121588 us; error 32 us; skew 500 ppm
I20260620 11:41:52.123152 25252 webserver.cc:460] Webserver started at http://127.5.122.190:46571/ using document root <none> and password file <none>
I20260620 11:41:52.123440 25252 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:52.123493 25252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:52.123664 25252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:52.124857 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "83ee25ce4a2e4432a34802b6cdcab1fa"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "40fd438231508d709d5dde6ea612198d"
server_key_iv: "8f5e41fc621482cf37ad0043580ec70b"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.125247 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "83ee25ce4a2e4432a34802b6cdcab1fa"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "40fd438231508d709d5dde6ea612198d"
server_key_iv: "8f5e41fc621482cf37ad0043580ec70b"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.126745 25252 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:41:52.127611 25266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.128016 25252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:52.128247 25252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "83ee25ce4a2e4432a34802b6cdcab1fa"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "40fd438231508d709d5dde6ea612198d"
server_key_iv: "8f5e41fc621482cf37ad0043580ec70b"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.128347 25252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:52.143780 25252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:52.144188 25252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:52.144366 25252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:52.148533 25252 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:36619
I20260620 11:41:52.148572 25318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:36619 every 8 connection(s)
I20260620 11:41:52.148998 25252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:52.150202 25319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:52.151311 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 25252
I20260620 11:41:52.151459 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:52.151666 5610 external_mini_cluster.cc:1428] Setting key 6ad769a81b7aa75ab777f4448c3833a7
I20260620 11:41:52.153019 25319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa: Bootstrap starting.
I20260620 11:41:52.153662 25319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:52.154048 25319 log.cc:824] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:52.154824 25319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa: No bootstrap required, opened a new log
I20260620 11:41:52.156692 25319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36619 } }
I20260620 11:41:52.156872 25319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:52.156919 25319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ee25ce4a2e4432a34802b6cdcab1fa, State: Initialized, Role: FOLLOWER
I20260620 11:41:52.157099 25319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36619 } }
I20260620 11:41:52.157197 25319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:52.157238 25319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:52.157290 25319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:52.157531 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.152277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.5.122.190:36619)
0620 11:41:52.152479 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.152487 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.152512 (+ 25us) server_negotiation.cc:407] Connection header received
0620 11:41:52.152935 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.152941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.153011 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.153188 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:52.153416 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.154498 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.155395 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.155848 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.155993 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.156753 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:52.156757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:52.156758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:52.156977 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:52.157025 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:52.157041 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:52.157097 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.157186 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.157275 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260620 11:41:52.158185 25319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36619 } }
I20260620 11:41:52.158327 25319 leader_election.cc:302] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83ee25ce4a2e4432a34802b6cdcab1fa; no voters:
I20260620 11:41:52.158516 25319 leader_election.cc:288] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:52.158552 25324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:52.158718 25324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [term 1 LEADER]: Becoming Leader. State: Replica: 83ee25ce4a2e4432a34802b6cdcab1fa, State: Running, Role: LEADER
I20260620 11:41:52.158812 25319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:52.158826 25324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36619 } }
I20260620 11:41:52.159386 25326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ee25ce4a2e4432a34802b6cdcab1fa. Latest consensus state: current_term: 1 leader_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36619 } } }
I20260620 11:41:52.159406 25325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ee25ce4a2e4432a34802b6cdcab1fa" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36619 } } }
I20260620 11:41:52.159490 25326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:52.159525 25325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:52.159745 25333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:52.160316 25333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:52.161931 25333 catalog_manager.cc:1347] Generated new cluster ID: ba4441e846f742e9a11f7cda2b7a4547
I20260620 11:41:52.162017 25333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:52.187248 25333 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:52.187814 25333 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:52.197041 25333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa: Generated new TSK 0
I20260620 11:41:52.197290 25333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:52.201640 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.129:0
--local_ip_for_outbound_sockets=127.5.122.129
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36619
--builtin_ntp_servers=127.5.122.148:34017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 11:41:52.288416 25343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:52.288763 25343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:52.288800 25343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:52.290171 25343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:52.290223 25343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:52.290252 25343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.129
I20260620 11:41:52.291821 25343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.122.129
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.129
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:52.292130 25343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:52.292512 25343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:52.298141 25348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:52.298134 25349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:52.298185 25343 server_base.cc:1034] running on GCE node
W20260620 11:41:52.298134 25351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:52.298604 25343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:52.298924 25343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:52.300156 25343 hybrid_clock.cc:648] HybridClock initialized: now 1781955712300123 us; error 46 us; skew 500 ppm
I20260620 11:41:52.301741 25343 webserver.cc:460] Webserver started at http://127.5.122.129:43449/ using document root <none> and password file <none>
I20260620 11:41:52.302124 25343 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:52.302222 25343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:52.302395 25343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:52.303581 25343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data/instance:
uuid: "97d30e27e6cb47108d5285c9979471e8"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "ebe03480dfbc938b8495889971727b59"
server_key_iv: "ce9aee09a392ad433aacc3e3368b4854"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.303994 25343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance:
uuid: "97d30e27e6cb47108d5285c9979471e8"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "ebe03480dfbc938b8495889971727b59"
server_key_iv: "ce9aee09a392ad433aacc3e3368b4854"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.305500 25343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:52.306497 25357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.306743 25343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:41:52.306879 25343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/wal
uuid: "97d30e27e6cb47108d5285c9979471e8"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "ebe03480dfbc938b8495889971727b59"
server_key_iv: "ce9aee09a392ad433aacc3e3368b4854"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.306975 25343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:52.321713 25343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:52.322206 25343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:52.322395 25343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:52.322674 25343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:52.323071 25343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:52.323136 25343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.323176 25343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:52.323211 25343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.330123 25343 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.129:42521
I20260620 11:41:52.330207 25470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.129:42521 every 8 connection(s)
I20260620 11:41:52.330632 25343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/data/info.pb
I20260620 11:41:52.336625 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.331593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:32899 (local address 127.5.122.190:36619)
0620 11:41:52.331684 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.331687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.331951 (+ 264us) server_negotiation.cc:407] Connection header received
0620 11:41:52.332538 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.332539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.332551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.332615 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:52.333412 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.334060 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.335133 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.335365 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.335429 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.336374 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:52.336375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:52.336375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:52.336381 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:52.336395 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:52.336399 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:52.336457 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.336529 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.336554 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 11:41:52.336848 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.331558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:36619 (local address 127.5.122.129:32899)
0620 11:41:52.331805 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.331843 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.332415 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.332747 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.332752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.332769 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:52.333309 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.333314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.334148 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.334149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.335040 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.335043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.335456 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.335457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.335578 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.336221 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:52.336221 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:52.336260 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:52.336287 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:52.336459 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:52.336462 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:52.336465 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.336517 (+ 52us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.336606 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":141,"threads_started":1}
I20260620 11:41:52.337423 25471 heartbeater.cc:347] Connected to a master server at 127.5.122.190:36619
I20260620 11:41:52.337533 25471 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:52.337724 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 25343
I20260620 11:41:52.337757 25471 heartbeater.cc:511] Master 127.5.122.190:36619 requested a full tablet report, sending...
I20260620 11:41:52.337838 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-0/wal/instance
I20260620 11:41:52.338184 5610 external_mini_cluster.cc:1428] Setting key c1ca1eaaf596b9a1aebfa2b35b585173
I20260620 11:41:52.338423 25283 ts_manager.cc:194] Registered new tserver with Master: 97d30e27e6cb47108d5285c9979471e8 (127.5.122.129:42521)
I20260620 11:41:52.339359 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.130:0
--local_ip_for_outbound_sockets=127.5.122.130
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36619
--builtin_ntp_servers=127.5.122.148:34017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 11:41:52.339413 25283 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.129:32899
W20260620 11:41:52.426321 25474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:52.426550 25474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:52.426574 25474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:52.427757 25474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:52.427802 25474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:52.427829 25474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.130
I20260620 11:41:52.429251 25474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.122.130
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.130
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:52.429476 25474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:52.429757 25474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:52.435256 25480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:52.435261 25479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:52.435331 25474 server_base.cc:1034] running on GCE node
W20260620 11:41:52.435261 25482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:52.435721 25474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:52.436033 25474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:52.437214 25474 hybrid_clock.cc:648] HybridClock initialized: now 1781955712437202 us; error 35 us; skew 500 ppm
I20260620 11:41:52.438735 25474 webserver.cc:460] Webserver started at http://127.5.122.130:36101/ using document root <none> and password file <none>
I20260620 11:41:52.439025 25474 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:52.439098 25474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:52.439265 25474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:52.440420 25474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data/instance:
uuid: "c099c24e5cef402caa2e9d49fc957b66"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "e7e8fb42fa18887de306e758d2d0bf63"
server_key_iv: "27e6e2db1ee82951767a8c90c5f93267"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.440841 25474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance:
uuid: "c099c24e5cef402caa2e9d49fc957b66"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "e7e8fb42fa18887de306e758d2d0bf63"
server_key_iv: "27e6e2db1ee82951767a8c90c5f93267"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.442363 25474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:41:52.443252 25488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.443457 25474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:52.443569 25474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal
uuid: "c099c24e5cef402caa2e9d49fc957b66"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "e7e8fb42fa18887de306e758d2d0bf63"
server_key_iv: "27e6e2db1ee82951767a8c90c5f93267"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.443650 25474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:52.454442 25474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:52.454802 25474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:52.454982 25474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:52.455248 25474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:52.455600 25474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:52.455659 25474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.455705 25474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:52.455734 25474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.462103 25474 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.130:34541
I20260620 11:41:52.462167 25601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.130:34541 every 8 connection(s)
I20260620 11:41:52.462560 25474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/data/info.pb
I20260620 11:41:52.465222 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 25474
I20260620 11:41:52.465337 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal/instance
I20260620 11:41:52.465535 5610 external_mini_cluster.cc:1428] Setting key cdc2d168d032a257c92ccd72f8fa9549
I20260620 11:41:52.466655 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.122.131:0
--local_ip_for_outbound_sockets=127.5.122.131
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36619
--builtin_ntp_servers=127.5.122.148:34017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 11:41:52.468789 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.463407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.130:33741 (local address 127.5.122.190:36619)
0620 11:41:52.463476 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.463478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.463782 (+ 304us) server_negotiation.cc:407] Connection header received
0620 11:41:52.464265 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.464267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.464277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.464332 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:52.465037 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.465850 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.467161 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.467488 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.467568 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.468523 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:52.468524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:52.468524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:52.468529 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:52.468546 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:52.468550 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:52.468595 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.468703 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.468727 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260620 11:41:52.469132 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.463411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:36619 (local address 127.5.122.130:33741)
0620 11:41:52.463623 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.463660 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.464180 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.464391 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.464394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.464404 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:52.464940 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.464945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.465966 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.465968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.467072 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.467076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.467579 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.467580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.467698 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.468346 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:52.468346 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:52.468394 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:52.468429 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:52.468614 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:52.468620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:52.468624 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.468701 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.468833 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260620 11:41:52.469657 25602 heartbeater.cc:347] Connected to a master server at 127.5.122.190:36619
I20260620 11:41:52.469781 25602 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:52.470038 25602 heartbeater.cc:511] Master 127.5.122.190:36619 requested a full tablet report, sending...
I20260620 11:41:52.470484 25283 ts_manager.cc:194] Registered new tserver with Master: c099c24e5cef402caa2e9d49fc957b66 (127.5.122.130:34541)
I20260620 11:41:52.471032 25283 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.130:33741
W20260620 11:41:52.552577 25605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:52.552814 25605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:52.552839 25605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:52.554157 25605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:52.554205 25605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:52.554234 25605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.122.131
I20260620 11:41:52.555701 25605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:34017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.122.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.122.131
--webserver_port=0
--tserver_master_addrs=127.5.122.190:36619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.122.131
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:52.555997 25605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:52.556340 25605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:52.562153 25610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:52.562312 25611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:52.562153 25613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:52.562469 25605 server_base.cc:1034] running on GCE node
I20260620 11:41:52.562726 25605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:52.562994 25605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:52.564150 25605 hybrid_clock.cc:648] HybridClock initialized: now 1781955712564136 us; error 37 us; skew 500 ppm
I20260620 11:41:52.565519 25605 webserver.cc:460] Webserver started at http://127.5.122.131:39329/ using document root <none> and password file <none>
I20260620 11:41:52.565855 25605 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:52.565907 25605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:52.566088 25605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:52.567253 25605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data/instance:
uuid: "9d0d4ebb438646dcaa1633aef2054550"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "b0e2788a3e8018edda28abe7e7d68f9c"
server_key_iv: "d1224f6b696023fbbc7cf32f158f642e"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.567633 25605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance:
uuid: "9d0d4ebb438646dcaa1633aef2054550"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "b0e2788a3e8018edda28abe7e7d68f9c"
server_key_iv: "d1224f6b696023fbbc7cf32f158f642e"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.569025 25605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:52.569859 25619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.570073 25605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:52.570188 25605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal
uuid: "9d0d4ebb438646dcaa1633aef2054550"
format_stamp: "Formatted at 2026-06-20 11:41:52 on dist-test-slave-szdr"
server_key: "b0e2788a3e8018edda28abe7e7d68f9c"
server_key_iv: "d1224f6b696023fbbc7cf32f158f642e"
server_key_version: "encryptionkey@0"
I20260620 11:41:52.570281 25605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:52.587491 25605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:52.587884 25605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:52.588045 25605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:52.588308 25605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 11:41:52.588672 25605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 11:41:52.588728 25605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.588778 25605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 11:41:52.588810 25605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:52.594950 25605 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.131:36439
I20260620 11:41:52.595033 25732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.131:36439 every 8 connection(s)
I20260620 11:41:52.595391 25605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/data/info.pb
I20260620 11:41:52.601080 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.596229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:52507 (local address 127.5.122.190:36619)
0620 11:41:52.596278 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.596280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.596516 (+ 236us) server_negotiation.cc:407] Connection header received
0620 11:41:52.597069 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.597070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.597082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.597153 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:52.597867 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.598430 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.599483 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.599689 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.599746 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.600803 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:52.600804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:52.600804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:52.600810 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:52.600825 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:52.600830 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:52.600886 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.600974 (+ 88us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.601006 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:41:52.601289 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.596218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:36619 (local address 127.5.122.131:52507)
0620 11:41:52.596402 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.596439 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.596961 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.597219 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.597223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.597231 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:41:52.597785 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.597790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.598523 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.598524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.599390 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.599392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.599776 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.599777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.599899 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.600561 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 11:41:52.600562 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:41:52.600596 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 11:41:52.600624 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:41:52.600898 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:41:52.600903 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:41:52.600905 (+ 2us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.600961 (+ 56us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.601055 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260620 11:41:52.601902 25733 heartbeater.cc:347] Connected to a master server at 127.5.122.190:36619
I20260620 11:41:52.602054 25733 heartbeater.cc:464] Registering TS with master...
I20260620 11:41:52.602268 25733 heartbeater.cc:511] Master 127.5.122.190:36619 requested a full tablet report, sending...
I20260620 11:41:52.602725 25283 ts_manager.cc:194] Registered new tserver with Master: 9d0d4ebb438646dcaa1633aef2054550 (127.5.122.131:36439)
I20260620 11:41:52.603138 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 25605
I20260620 11:41:52.603173 25283 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.122.131:52507
I20260620 11:41:52.603300 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal/instance
I20260620 11:41:52.603547 5610 external_mini_cluster.cc:1428] Setting key 9ac852a014aa32c7f00281cdcdfca5b6
I20260620 11:41:52.604471 5610 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 11:41:52.608760 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.605825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.5.122.190:36619)
0620 11:41:52.605905 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.605908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.606010 (+ 102us) server_negotiation.cc:407] Connection header received
0620 11:41:52.606095 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.606096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.606104 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.606154 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:52.606317 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.606795 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.607549 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.607743 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.607819 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.608520 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:52.608521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:52.608521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:52.608526 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:52.608538 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:52.608542 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:52.608610 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.608651 (+ 41us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.608671 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 11:41:52.610523 25283 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 11:41:52.612282 25283 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 11:41:52.619855 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.614993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.5.122.131:36439)
0620 11:41:52.615591 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.615596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.615607 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:41:52.615644 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.615646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.615663 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.615774 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:52.615867 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.617026 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.618731 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.619505 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.619576 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.619681 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.619712 (+ 31us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.619760 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":102,"threads_started":1}
I20260620 11:41:52.620097 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.614906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.5.122.130:34541)
0620 11:41:52.615187 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.615193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.615281 (+ 88us) server_negotiation.cc:407] Connection header received
0620 11:41:52.615603 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.615605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.615620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.615690 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:52.615910 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.617026 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.618752 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.619324 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.619411 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.619513 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.619958 (+ 445us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.620004 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
I20260620 11:41:52.619902 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.614797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:36439 (local address 127.0.0.1:33264)
0620 11:41:52.615346 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.615356 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.615428 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.615772 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.615772 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.615774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:52.615810 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.615811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.617129 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.617130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.618632 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.618635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.619578 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.619579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.619645 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.619649 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.619693 (+ 44us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.619761 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":59,"thread_start_us":82,"threads_started":1}
I20260620 11:41:52.619922 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.614797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:34541 (local address 127.0.0.1:33442)
0620 11:41:52.615166 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.615219 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.615511 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.615707 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.615711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.615713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:52.615817 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.615820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.617129 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.617130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.618632 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.618635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.619411 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.619412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.619495 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.619509 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.619768 (+ 259us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.619813 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":142,"thread_start_us":80,"threads_started":1}
I20260620 11:41:52.620793 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.615025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:42521 (local address 127.0.0.1:44846)
0620 11:41:52.615364 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.615374 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.615431 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.615709 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.615712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.615714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:52.615808 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.615810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.618223 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.618224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.619235 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.619238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.620546 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.620548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.620620 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.620624 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.620685 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.620726 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260620 11:41:52.620823 25746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.615132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.5.122.129:42521)
0620 11:41:52.615312 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.615317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.615421 (+ 104us) server_negotiation.cc:407] Connection header received
0620 11:41:52.615518 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.615520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.615538 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.615612 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:52.617146 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.618138 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.619346 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.620308 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.620384 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.620475 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.620703 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.620741 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260620 11:41:52.621145 25536 tablet_service.cc:1467] Processing CreateTablet for tablet 794ba0288f3b4762a5dd337bdd896b59 (DEFAULT_TABLE table=test-table [id=782516c218f74b2d84ecd2a2df58080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:52.621142 25667 tablet_service.cc:1467] Processing CreateTablet for tablet 794ba0288f3b4762a5dd337bdd896b59 (DEFAULT_TABLE table=test-table [id=782516c218f74b2d84ecd2a2df58080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:52.621548 25667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794ba0288f3b4762a5dd337bdd896b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:52.621548 25536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794ba0288f3b4762a5dd337bdd896b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:52.621666 25405 tablet_service.cc:1467] Processing CreateTablet for tablet 794ba0288f3b4762a5dd337bdd896b59 (DEFAULT_TABLE table=test-table [id=782516c218f74b2d84ecd2a2df58080b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 11:41:52.622051 25405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 794ba0288f3b4762a5dd337bdd896b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:52.624862 25747 tablet_bootstrap.cc:492] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: Bootstrap starting.
I20260620 11:41:52.625124 25748 tablet_bootstrap.cc:492] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Bootstrap starting.
I20260620 11:41:52.625638 25747 tablet_bootstrap.cc:654] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:52.625741 25748 tablet_bootstrap.cc:654] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:52.626117 25748 log.cc:824] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:52.626317 25747 log.cc:824] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:52.626526 25749 tablet_bootstrap.cc:492] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Bootstrap starting.
I20260620 11:41:52.627113 25747 tablet_bootstrap.cc:492] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: No bootstrap required, opened a new log
I20260620 11:41:52.627195 25747 ts_tablet_manager.cc:1397] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260620 11:41:52.627223 25749 tablet_bootstrap.cc:654] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:52.627391 25748 tablet_bootstrap.cc:492] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: No bootstrap required, opened a new log
I20260620 11:41:52.627547 25749 log.cc:824] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:52.627565 25748 ts_tablet_manager.cc:1397] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260620 11:41:52.628233 25749 tablet_bootstrap.cc:492] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: No bootstrap required, opened a new log
I20260620 11:41:52.628317 25749 ts_tablet_manager.cc:1397] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260620 11:41:52.629902 25747 raft_consensus.cc:348] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.630292 25747 raft_consensus.cc:374] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:52.630225 25748 raft_consensus.cc:348] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.630352 25747 raft_consensus.cc:729] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d30e27e6cb47108d5285c9979471e8, State: Initialized, Role: FOLLOWER
I20260620 11:41:52.630386 25748 raft_consensus.cc:374] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:52.630270 25749 raft_consensus.cc:348] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.630416 25748 raft_consensus.cc:729] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0d4ebb438646dcaa1633aef2054550, State: Initialized, Role: FOLLOWER
I20260620 11:41:52.630432 25749 raft_consensus.cc:374] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:52.630472 25749 raft_consensus.cc:729] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c099c24e5cef402caa2e9d49fc957b66, State: Initialized, Role: FOLLOWER
I20260620 11:41:52.630534 25748 consensus_queue.cc:260] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.630534 25747 consensus_queue.cc:260] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.630622 25749 consensus_queue.cc:260] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.630827 25748 ts_tablet_manager.cc:1428] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:52.630827 25747 ts_tablet_manager.cc:1428] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260620 11:41:52.630885 25749 ts_tablet_manager.cc:1428] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 11:41:52.630967 25733 heartbeater.cc:503] Master 127.5.122.190:36619 was elected leader, sending a full tablet report...
I20260620 11:41:52.631023 25602 heartbeater.cc:503] Master 127.5.122.190:36619 was elected leader, sending a full tablet report...
I20260620 11:41:52.631301 25471 heartbeater.cc:503] Master 127.5.122.190:36619 was elected leader, sending a full tablet report...
W20260620 11:41:52.713186 25603 tablet.cc:2369] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:41:52.831486 25472 tablet.cc:2369] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 11:41:52.846120 25734 tablet.cc:2369] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 11:41:52.867344 25754 raft_consensus.cc:482] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:52.867497 25754 raft_consensus.cc:504] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.867846 25754 leader_election.cc:288] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c099c24e5cef402caa2e9d49fc957b66 (127.5.122.130:34541), 9d0d4ebb438646dcaa1633aef2054550 (127.5.122.131:36439)
I20260620 11:41:52.869536 25753 raft_consensus.cc:482] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 11:41:52.869676 25753 raft_consensus.cc:504] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.870098 25753 leader_election.cc:288] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c099c24e5cef402caa2e9d49fc957b66 (127.5.122.130:34541), 97d30e27e6cb47108d5285c9979471e8 (127.5.122.129:42521)
I20260620 11:41:52.871738 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.868181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:33467 (local address 127.5.122.130:34541)
0620 11:41:52.868234 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.868236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.868357 (+ 121us) server_negotiation.cc:407] Connection header received
0620 11:41:52.868486 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.868487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.868495 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.868564 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:52.868758 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.869237 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.870719 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.871285 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.871332 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.871340 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.871559 (+ 219us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.871599 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:41:52.871773 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.868255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.129:60911 (local address 127.5.122.131:36439)
0620 11:41:52.868310 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.868313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.868383 (+ 70us) server_negotiation.cc:407] Connection header received
0620 11:41:52.868496 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.868497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.868505 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.868582 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:52.868762 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.869243 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.870718 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.871304 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.871345 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.871352 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.871561 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.871599 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260620 11:41:52.872087 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.868165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.131:36439 (local address 127.5.122.129:60911)
0620 11:41:52.868291 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.868301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.868395 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.868595 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.868596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.868598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:52.868682 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.868683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.869332 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.869333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.870621 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.870624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.871417 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.871418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.871485 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.871489 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.871852 (+ 363us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.871905 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":50,"threads_started":1}
I20260620 11:41:52.872097 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.868115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:34541 (local address 127.5.122.129:33467)
0620 11:41:52.868275 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.868287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.868371 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.868583 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.868584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.868586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:52.868683 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.868685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.869331 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.869333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.870623 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.870626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.871412 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.871412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.871482 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.871486 (+ 4us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.871887 (+ 401us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.871935 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":59,"thread_start_us":66,"threads_started":1}
I20260620 11:41:52.872540 25687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ba0288f3b4762a5dd337bdd896b59" candidate_uuid: "97d30e27e6cb47108d5285c9979471e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d0d4ebb438646dcaa1633aef2054550" is_pre_election: true
I20260620 11:41:52.872665 25687 raft_consensus.cc:2454] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d30e27e6cb47108d5285c9979471e8 in term 0.
I20260620 11:41:52.872929 25556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ba0288f3b4762a5dd337bdd896b59" candidate_uuid: "97d30e27e6cb47108d5285c9979471e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c099c24e5cef402caa2e9d49fc957b66" is_pre_election: true
I20260620 11:41:52.872993 25358 leader_election.cc:302] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d30e27e6cb47108d5285c9979471e8, 9d0d4ebb438646dcaa1633aef2054550; no voters:
I20260620 11:41:52.873090 25556 raft_consensus.cc:2454] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d30e27e6cb47108d5285c9979471e8 in term 0.
I20260620 11:41:52.873122 25754 raft_consensus.cc:2781] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 11:41:52.873163 25754 raft_consensus.cc:482] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 11:41:52.873193 25754 raft_consensus.cc:3037] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:52.874058 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.870264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.130:34541 (local address 127.5.122.131:56717)
0620 11:41:52.870486 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.870497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.870540 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.870760 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.870761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.870763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:52.870838 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.870839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.871744 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.871745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.872953 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.872955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.873776 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.873778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.873844 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.873847 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.873925 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.873991 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260620 11:41:52.874069 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.870306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:56717 (local address 127.5.122.130:34541)
0620 11:41:52.870470 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.870472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.870535 (+ 63us) server_negotiation.cc:407] Connection header received
0620 11:41:52.870607 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.870608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.870618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.870898 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:52.870928 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.871623 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.873059 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.873667 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.873756 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.873765 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.873931 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.873991 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260620 11:41:52.874217 25754 raft_consensus.cc:504] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.874372 25754 leader_election.cc:288] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [CANDIDATE]: Term 1 election: Requested vote from peers c099c24e5cef402caa2e9d49fc957b66 (127.5.122.130:34541), 9d0d4ebb438646dcaa1633aef2054550 (127.5.122.131:36439)
I20260620 11:41:52.874382 25556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ba0288f3b4762a5dd337bdd896b59" candidate_uuid: "9d0d4ebb438646dcaa1633aef2054550" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c099c24e5cef402caa2e9d49fc957b66" is_pre_election: true
I20260620 11:41:52.874475 25556 raft_consensus.cc:2454] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d0d4ebb438646dcaa1633aef2054550 in term 0.
I20260620 11:41:52.874564 25555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ba0288f3b4762a5dd337bdd896b59" candidate_uuid: "97d30e27e6cb47108d5285c9979471e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c099c24e5cef402caa2e9d49fc957b66"
I20260620 11:41:52.874610 25746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.870651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.122.131:35421 (local address 127.5.122.129:42521)
0620 11:41:52.870712 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.870713 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.870837 (+ 124us) server_negotiation.cc:407] Connection header received
0620 11:41:52.870922 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.870923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.870932 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.870959 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 11:41:52.871134 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.871823 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.873448 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.873989 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.874057 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.874065 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.874507 (+ 442us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.874549 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:41:52.874622 25555 raft_consensus.cc:3037] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:52.874609 25687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ba0288f3b4762a5dd337bdd896b59" candidate_uuid: "97d30e27e6cb47108d5285c9979471e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d0d4ebb438646dcaa1633aef2054550"
I20260620 11:41:52.874644 25622 leader_election.cc:302] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d0d4ebb438646dcaa1633aef2054550, c099c24e5cef402caa2e9d49fc957b66; no voters:
I20260620 11:41:52.874682 25687 raft_consensus.cc:3037] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:52.874789 25759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.870366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:42521 (local address 127.5.122.131:35421)
0620 11:41:52.870782 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.870791 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:41:52.870858 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:41:52.871011 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:41:52.871011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:41:52.871013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 11:41:52.871069 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.871070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.872217 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.872218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.873370 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:41:52.873372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.874330 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.874331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:41:52.874405 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.874408 (+ 3us) client_negotiation.cc:769] Sending connection context
0620 11:41:52.874668 (+ 260us) client_negotiation.cc:240] Negotiation successful
0620 11:41:52.874712 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":75,"threads_started":1}
I20260620 11:41:52.875427 25687 raft_consensus.cc:2454] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d30e27e6cb47108d5285c9979471e8 in term 1.
I20260620 11:41:52.875530 25555 raft_consensus.cc:2454] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d30e27e6cb47108d5285c9979471e8 in term 1.
I20260620 11:41:52.875628 25753 raft_consensus.cc:2741] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260620 11:41:52.875648 25358 leader_election.cc:302] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d30e27e6cb47108d5285c9979471e8, 9d0d4ebb438646dcaa1633aef2054550; no voters:
I20260620 11:41:52.875730 25754 raft_consensus.cc:2781] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:52.875747 25425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "794ba0288f3b4762a5dd337bdd896b59" candidate_uuid: "9d0d4ebb438646dcaa1633aef2054550" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d30e27e6cb47108d5285c9979471e8" is_pre_election: true
I20260620 11:41:52.875869 25754 raft_consensus.cc:686] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 1 LEADER]: Becoming Leader. State: Replica: 97d30e27e6cb47108d5285c9979471e8, State: Running, Role: LEADER
I20260620 11:41:52.875975 25754 consensus_queue.cc:237] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } }
I20260620 11:41:52.876889 25282 catalog_manager.cc:5495] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d30e27e6cb47108d5285c9979471e8 (127.5.122.129). New cstate: current_term: 1 leader_uuid: "97d30e27e6cb47108d5285c9979471e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d30e27e6cb47108d5285c9979471e8" member_type: VOTER last_known_addr { host: "127.5.122.129" port: 42521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 } health_report { overall_health: UNKNOWN } } }
I20260620 11:41:52.902611 25746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.899822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.5.122.129:42521)
0620 11:41:52.899890 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:52.899892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:52.899899 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:41:52.899936 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:52.899937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:52.900030 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:52.900127 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:41:52.900293 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.900848 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.901663 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:52.901865 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:52.901989 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:52.902186 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:41:52.902310 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:41:52.902419 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:52.902452 (+ 33us) server_negotiation.cc:299] Negotiation successful
0620 11:41:52.902479 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:41:52.904838 25687 raft_consensus.cc:1261] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 1 FOLLOWER]: Refusing update from remote peer 97d30e27e6cb47108d5285c9979471e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:52.904840 25555 raft_consensus.cc:1261] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 1 FOLLOWER]: Refusing update from remote peer 97d30e27e6cb47108d5285c9979471e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:41:52.905256 25754 consensus_queue.cc:1035] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d0d4ebb438646dcaa1633aef2054550" member_type: VOTER last_known_addr { host: "127.5.122.131" port: 36439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:52.905431 25760 consensus_queue.cc:1035] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c099c24e5cef402caa2e9d49fc957b66" member_type: VOTER last_known_addr { host: "127.5.122.130" port: 34541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:41:52.908062 25763 mvcc.cc:205] Tried to move back new op lower bound from 7298890600054505472 to 7298890599940730880. Current Snapshot: MvccSnapshot[applied={T|T < 7298890600054505472}]
I20260620 11:41:52.908404 25764 mvcc.cc:205] Tried to move back new op lower bound from 7298890600054505472 to 7298890599940730880. Current Snapshot: MvccSnapshot[applied={T|T < 7298890600054505472}]
I20260620 11:41:52.911159 25282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:52.911264 25282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 11:41:52.912166 25282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83ee25ce4a2e4432a34802b6cdcab1fa: Sending DeleteTablet for 3 replicas of tablet 794ba0288f3b4762a5dd337bdd896b59
I20260620 11:41:52.912601 25667 tablet_service.cc:1514] Processing DeleteTablet for tablet 794ba0288f3b4762a5dd337bdd896b59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:52 UTC) from {username='slave'} at 127.0.0.1:33264
I20260620 11:41:52.912699 25536 tablet_service.cc:1514] Processing DeleteTablet for tablet 794ba0288f3b4762a5dd337bdd896b59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:52 UTC) from {username='slave'} at 127.0.0.1:33442
I20260620 11:41:52.912726 25405 tablet_service.cc:1514] Processing DeleteTablet for tablet 794ba0288f3b4762a5dd337bdd896b59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 11:41:52 UTC) from {username='slave'} at 127.0.0.1:44846
I20260620 11:41:52.912961 25775 tablet_replica.cc:330] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8: stopping tablet replica
I20260620 11:41:52.913064 25775 raft_consensus.cc:2229] T 794ba0288f3b4762a5dd337bdd896b59 P 97d30e27e6cb47108d5285c9979471e8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 11:41:52.913110 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 25343
I20260620 11:41:52.914016 25773 tablet_replica.cc:330] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: stopping tablet replica
I20260620 11:41:52.914229 25773 raft_consensus.cc:2229] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:52.914433 25773 raft_consensus.cc:2258] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:52.914959 25773 ts_tablet_manager.cc:1905] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:52.915598 25774 tablet_replica.cc:330] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: stopping tablet replica
I20260620 11:41:52.915733 25774 raft_consensus.cc:2229] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 11:41:52.915849 25774 raft_consensus.cc:2258] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 11:41:52.916358 25774 ts_tablet_manager.cc:1905] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 11:41:52.916894 25773 ts_tablet_manager.cc:1918] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:52.916992 25773 log.cc:1196] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-2/wal/wals/794ba0288f3b4762a5dd337bdd896b59
I20260620 11:41:52.917646 25773 ts_tablet_manager.cc:1939] T 794ba0288f3b4762a5dd337bdd896b59 P 9d0d4ebb438646dcaa1633aef2054550: Deleting consensus metadata
I20260620 11:41:52.918154 25267 catalog_manager.cc:4841] TS 9d0d4ebb438646dcaa1633aef2054550 (127.5.122.131:36439): tablet 794ba0288f3b4762a5dd337bdd896b59 (table test-table [id=782516c218f74b2d84ecd2a2df58080b]) successfully deleted
I20260620 11:41:52.918505 25774 ts_tablet_manager.cc:1918] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 11:41:52.918593 25774 log.cc:1196] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Deleting WAL directory at /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781955466149030-5610-0/minicluster-data/ts-1/wal/wals/794ba0288f3b4762a5dd337bdd896b59
I20260620 11:41:52.918845 25774 ts_tablet_manager.cc:1939] T 794ba0288f3b4762a5dd337bdd896b59 P c099c24e5cef402caa2e9d49fc957b66: Deleting consensus metadata
I20260620 11:41:52.919523 25269 catalog_manager.cc:4841] TS c099c24e5cef402caa2e9d49fc957b66 (127.5.122.130:34541): tablet 794ba0288f3b4762a5dd337bdd896b59 (table test-table [id=782516c218f74b2d84ecd2a2df58080b]) successfully deleted
W20260620 11:41:52.919778 25269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.122.129:42521 (error 108)
I20260620 11:41:52.920043 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 25474
I20260620 11:41:52.920826 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:52.920680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.129:42521 (local address 127.0.0.1:44864)
0620 11:41:52.920735 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 11:41:52.920783 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.129:42521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 11:41:52.920946 25269 catalog_manager.cc:4584] TS 97d30e27e6cb47108d5285c9979471e8 (127.5.122.129:42521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 794ba0288f3b4762a5dd337bdd896b59: Network error: Client connection negotiation failed: client connection to 127.5.122.129:42521: connect: Connection refused (error 111)
I20260620 11:41:52.924724 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 25605
I20260620 11:41:52.929869 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 25252
2026-06-20T11:41:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (932 ms)
[----------] 6 tests from EncryptionPolicyTest (8867 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:52 dist-test-slave-szdr krb5kdc[25779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:52 dist-test-slave-szdr krb5kdc[25779](info): set up 2 sockets
Jun 20 11:41:52 dist-test-slave-szdr krb5kdc[25779](info): commencing operation
krb5kdc: starting...
W20260620 11:41:53.974606 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:54 dist-test-slave-szdr krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:41:54.104712 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:54.189248 25792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:54.189559 25792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:54.189604 25792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:54.191098 25792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:54.191210 25792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:54.191236 25792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:54.191259 25792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:54.191291 25792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:54.193115 25792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44703
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:54.193500 25792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:54.193776 25792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:54.199506 25798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:54.199654 25792 server_base.cc:1034] running on GCE node
W20260620 11:41:54.199522 25800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:54.199553 25797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:54.200234 25792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:54.200574 25792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:54.201771 25792 hybrid_clock.cc:648] HybridClock initialized: now 1781955714201757 us; error 53 us; skew 500 ppm
Jun 20 11:41:54 dist-test-slave-szdr krb5kdc[25779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955714, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:54.208648 25792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:41:54.209277 25792 webserver.cc:460] Webserver started at http://127.0.0.1:45521/ using document root <none> and password file <none>
I20260620 11:41:54.209544 25792 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:54.209620 25792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:54.209786 25792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:54.211189 25792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "b95c11f9562b4f7291def4cce43aa2a2"
format_stamp: "Formatted at 2026-06-20 11:41:54 on dist-test-slave-szdr"
server_key: "043ae13778116a0aec2e19db3cfcd7db"
server_key_iv: "e7c7184e033e53a1f4755af287b42721"
server_key_version: "encryptionkey@0"
I20260620 11:41:54.211608 25792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "b95c11f9562b4f7291def4cce43aa2a2"
format_stamp: "Formatted at 2026-06-20 11:41:54 on dist-test-slave-szdr"
server_key: "043ae13778116a0aec2e19db3cfcd7db"
server_key_iv: "e7c7184e033e53a1f4755af287b42721"
server_key_version: "encryptionkey@0"
I20260620 11:41:54.213163 25792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:41:54.214115 25807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:54.214316 25792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:54.214432 25792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "b95c11f9562b4f7291def4cce43aa2a2"
format_stamp: "Formatted at 2026-06-20 11:41:54 on dist-test-slave-szdr"
server_key: "043ae13778116a0aec2e19db3cfcd7db"
server_key_iv: "e7c7184e033e53a1f4755af287b42721"
server_key_version: "encryptionkey@0"
I20260620 11:41:54.214529 25792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:54.237421 25792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:54.239877 25792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:54.240091 25792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:54.244570 25792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44703
I20260620 11:41:54.244630 25859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44703 every 8 connection(s)
I20260620 11:41:54.245055 25792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:54.246227 25860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:54.248790 25860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2: Bootstrap starting.
I20260620 11:41:54.249439 25860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:54.249707 25860 log.cc:824] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:54.250610 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 25792
I20260620 11:41:54.250658 25860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2: No bootstrap required, opened a new log
I20260620 11:41:54.250768 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:54.251027 5610 external_mini_cluster.cc:1428] Setting key 2e10cb1d523b4020c60433f116d6fdf1
I20260620 11:41:54.253127 25860 raft_consensus.cc:348] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95c11f9562b4f7291def4cce43aa2a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260620 11:41:54.253340 25860 raft_consensus.cc:374] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:54.253386 25860 raft_consensus.cc:729] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b95c11f9562b4f7291def4cce43aa2a2, State: Initialized, Role: FOLLOWER
I20260620 11:41:54.253592 25860 consensus_queue.cc:260] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95c11f9562b4f7291def4cce43aa2a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260620 11:41:54.253705 25860 raft_consensus.cc:388] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:54.253747 25860 raft_consensus.cc:482] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:54.253799 25860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:54.255125 25860 raft_consensus.cc:504] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95c11f9562b4f7291def4cce43aa2a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260620 11:41:54.255239 25860 leader_election.cc:302] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b95c11f9562b4f7291def4cce43aa2a2; no voters:
I20260620 11:41:54.255502 25860 leader_election.cc:288] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:54.255525 25865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:54.255651 25865 raft_consensus.cc:686] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [term 1 LEADER]: Becoming Leader. State: Replica: b95c11f9562b4f7291def4cce43aa2a2, State: Running, Role: LEADER
I20260620 11:41:54.255833 25860 sys_catalog.cc:564] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:54.255831 25865 consensus_queue.cc:237] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95c11f9562b4f7291def4cce43aa2a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } }
I20260620 11:41:54.256394 25867 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b95c11f9562b4f7291def4cce43aa2a2. Latest consensus state: current_term: 1 leader_uuid: "b95c11f9562b4f7291def4cce43aa2a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95c11f9562b4f7291def4cce43aa2a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } } }
I20260620 11:41:54.256501 25867 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:54.256762 25866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b95c11f9562b4f7291def4cce43aa2a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b95c11f9562b4f7291def4cce43aa2a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44703 } } }
I20260620 11:41:54.256908 25866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:54.257078 25872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:54.257742 25872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 11:41:54 dist-test-slave-szdr krb5kdc[25779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 11:41:54.259982 25872 catalog_manager.cc:1347] Generated new cluster ID: cecd9c52f4a9480a9f0646cb8248bbc3
I20260620 11:41:54.260028 25872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:54.265059 25864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:54.251702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43473 (local address 127.0.0.1:44703)
0620 11:41:54.251946 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:54.251958 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:54.252195 (+ 237us) server_negotiation.cc:407] Connection header received
0620 11:41:54.253111 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:54.253121 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:54.253186 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:54.253369 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:54.254430 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:54.255662 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:54.256780 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:54.257431 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:54.257543 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:54.260546 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:54.260551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:54.260552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:54.260567 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:54.263011 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:54.263549 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:54.263553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:54.263554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:54.263696 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:54.264010 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:54.264011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:54.264011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:54.264418 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:54.264625 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:54.264714 (+ 89us) server_negotiation.cc:299] Negotiation successful
0620 11:41:54.264832 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":119,"threads_started":1}
I20260620 11:41:54.266335 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:41:54.273880 25872 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:54.274480 25872 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:54.275931 25864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:54.267878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47325 (local address 127.0.0.1:44703)
0620 11:41:54.267952 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:54.267954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:54.268089 (+ 135us) server_negotiation.cc:407] Connection header received
0620 11:41:54.268136 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:54.268136 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:54.268148 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:54.268212 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:54.268869 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:54.269671 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:54.270781 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:54.271119 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:54.271202 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:54.272762 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:54.272763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:54.272764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:54.272772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:54.274515 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:54.274999 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:54.275000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:54.275001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:54.275068 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:54.275315 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:54.275315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:54.275316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:54.275436 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:54.275542 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:54.275764 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 11:41:54.275835 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":49}
I20260620 11:41:54.280085 25872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b95c11f9562b4f7291def4cce43aa2a2: Generated new TSK 0
I20260620 11:41:54.280319 25872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:54.284015 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 25792
W20260620 11:41:54.288978 25777 connection.cc:535] client connection to 127.0.0.1:44703 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44703): Connection reset by peer (error 104)
2026-06-20T11:41:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:54 dist-test-slave-szdr krb5kdc[25891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:54 dist-test-slave-szdr krb5kdc[25891](info): set up 2 sockets
Jun 20 11:41:54 dist-test-slave-szdr krb5kdc[25891](info): commencing operation
krb5kdc: starting...
W20260620 11:41:55.394286 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:55 dist-test-slave-szdr krb5kdc[25891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:41:55.524791 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:55.610821 25904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:55.611063 25904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:55.611091 25904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:55.612392 25904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:55.612447 25904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:55.612461 25904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:55.612474 25904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:55.612488 25904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:55.614151 25904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:55.614461 25904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:55.614727 25904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:55.620110 25909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:55.620244 25904 server_base.cc:1034] running on GCE node
W20260620 11:41:55.620087 25910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:55.620092 25912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:55.620620 25904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:55.620967 25904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:55.622140 25904 hybrid_clock.cc:648] HybridClock initialized: now 1781955715622101 us; error 43 us; skew 500 ppm
Jun 20 11:41:55 dist-test-slave-szdr krb5kdc[25891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:55.624982 25904 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:41:55.625547 25904 webserver.cc:460] Webserver started at http://127.0.0.1:33563/ using document root <none> and password file <none>
I20260620 11:41:55.625803 25904 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:55.625875 25904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:55.626072 25904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:55.627204 25904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "5c5cf567e00c40afae17fa7e5ffa54d0"
format_stamp: "Formatted at 2026-06-20 11:41:55 on dist-test-slave-szdr"
server_key: "0d2a5120b3abd4dd24cf26d1d24e5408"
server_key_iv: "fef9c3e3c979130dcf27db82420cccb5"
server_key_version: "encryptionkey@0"
I20260620 11:41:55.627581 25904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "5c5cf567e00c40afae17fa7e5ffa54d0"
format_stamp: "Formatted at 2026-06-20 11:41:55 on dist-test-slave-szdr"
server_key: "0d2a5120b3abd4dd24cf26d1d24e5408"
server_key_iv: "fef9c3e3c979130dcf27db82420cccb5"
server_key_version: "encryptionkey@0"
I20260620 11:41:55.629045 25904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:41:55.630076 25919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:55.630291 25904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:55.630411 25904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "5c5cf567e00c40afae17fa7e5ffa54d0"
format_stamp: "Formatted at 2026-06-20 11:41:55 on dist-test-slave-szdr"
server_key: "0d2a5120b3abd4dd24cf26d1d24e5408"
server_key_iv: "fef9c3e3c979130dcf27db82420cccb5"
server_key_version: "encryptionkey@0"
I20260620 11:41:55.630497 25904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:55.642841 25904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:55.645040 25904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:55.645260 25904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:55.649427 25904 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44165
I20260620 11:41:55.649462 25971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44165 every 8 connection(s)
I20260620 11:41:55.649849 25904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:55.650197 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 25904
I20260620 11:41:55.650321 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:55.650503 5610 external_mini_cluster.cc:1428] Setting key 27007b0a9981fef70ee50cfbf8647e22
I20260620 11:41:55.651044 25972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:55.654537 25972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0: Bootstrap starting.
I20260620 11:41:55.655265 25972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:55.655607 25972 log.cc:824] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:55.656500 25972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0: No bootstrap required, opened a new log
I20260620 11:41:55.657037 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:55.651052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41163 (local address 127.0.0.1:44165)
0620 11:41:55.651244 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:55.651254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:55.651289 (+ 35us) server_negotiation.cc:407] Connection header received
0620 11:41:55.651895 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:55.651905 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:55.651979 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:55.652148 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:55.652395 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:55.653813 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:55.654777 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:55.655393 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:55.655515 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:55.656310 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:55.656314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:55.656315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:55.656524 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:55.656565 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:55.656581 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:55.656688 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:55.656779 (+ 91us) server_negotiation.cc:299] Negotiation successful
0620 11:41:55.656808 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260620 11:41:55.658282 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:41:55.658990 25972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260620 11:41:55.659205 25972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:55.659242 25972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5cf567e00c40afae17fa7e5ffa54d0, State: Initialized, Role: FOLLOWER
I20260620 11:41:55.659368 25972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260620 11:41:55.659466 25972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:55.659516 25972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:55.659583 25972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:55.660506 25972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260620 11:41:55.660622 25972 leader_election.cc:302] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c5cf567e00c40afae17fa7e5ffa54d0; no voters:
I20260620 11:41:55.660832 25972 leader_election.cc:288] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:55.660861 25982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:55.660972 25982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [term 1 LEADER]: Becoming Leader. State: Replica: 5c5cf567e00c40afae17fa7e5ffa54d0, State: Running, Role: LEADER
I20260620 11:41:55.661064 25982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } }
I20260620 11:41:55.661113 25972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:55.661402 25982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } } }
I20260620 11:41:55.661494 25982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:55.661670 25982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c5cf567e00c40afae17fa7e5ffa54d0. Latest consensus state: current_term: 1 leader_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c5cf567e00c40afae17fa7e5ffa54d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44165 } } }
I20260620 11:41:55.661768 25986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:55.661844 25982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:55.662439 25986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:55.664062 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:55.660055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39307 (local address 127.0.0.1:44165)
0620 11:41:55.660099 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:55.660102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:55.660236 (+ 134us) server_negotiation.cc:407] Connection header received
0620 11:41:55.660298 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:55.660298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:55.660307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:55.660354 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:55.660491 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:55.661307 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:55.662454 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:55.662718 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:55.662784 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:55.663754 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:55.663755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:55.663756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:55.663760 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:55.663777 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:55.663782 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:55.663883 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:55.663973 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 11:41:55.663984 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:41:55.664731 25986 catalog_manager.cc:1347] Generated new cluster ID: f63802033ca4421bb2036539aafbaf2d
I20260620 11:41:55.664785 25986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:55.674922 25986 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:55.675520 25986 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:55.678264 25986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c5cf567e00c40afae17fa7e5ffa54d0: Generated new TSK 0
I20260620 11:41:55.678486 25986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:55.681587 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 25904
W20260620 11:41:55.688598 25889 connection.cc:535] client connection to 127.0.0.1:44165 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44165): Connection reset by peer (error 104)
2026-06-20T11:41:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:55 dist-test-slave-szdr krb5kdc[26003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:55 dist-test-slave-szdr krb5kdc[26003](info): set up 2 sockets
Jun 20 11:41:55 dist-test-slave-szdr krb5kdc[26003](info): commencing operation
krb5kdc: starting...
W20260620 11:41:56.722067 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:56 dist-test-slave-szdr krb5kdc[26003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:41:56.853654 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33287
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:56.939718 26016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:56.939982 26016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:56.940011 26016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:56.941246 26016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:56.941293 26016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:56.941308 26016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:56.941321 26016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:56.941334 26016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:56.942978 26016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33287
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:56.943219 26016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:56.943421 26016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:56.948904 26022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:56.948915 26024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:56.948904 26021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:56.949244 26016 server_base.cc:1034] running on GCE node
I20260620 11:41:56.949637 26016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:56.950030 26016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:56.951228 26016 hybrid_clock.cc:648] HybridClock initialized: now 1781955716951192 us; error 56 us; skew 500 ppm
Jun 20 11:41:56 dist-test-slave-szdr krb5kdc[26003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:56.954376 26016 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:41:56.954944 26016 webserver.cc:460] Webserver started at http://127.0.0.1:37961/ using document root <none> and password file <none>
I20260620 11:41:56.955235 26016 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:56.955312 26016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:56.955480 26016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:56.956784 26016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "fd4d8a0299844fed9345cf5201d54aa9"
format_stamp: "Formatted at 2026-06-20 11:41:56 on dist-test-slave-szdr"
server_key: "b80411076ff6f8b6e86a5d794a726ce2"
server_key_iv: "7423bc4800f34c18d32a7fc3f49c1117"
server_key_version: "encryptionkey@0"
I20260620 11:41:56.957218 26016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "fd4d8a0299844fed9345cf5201d54aa9"
format_stamp: "Formatted at 2026-06-20 11:41:56 on dist-test-slave-szdr"
server_key: "b80411076ff6f8b6e86a5d794a726ce2"
server_key_iv: "7423bc4800f34c18d32a7fc3f49c1117"
server_key_version: "encryptionkey@0"
I20260620 11:41:56.958918 26016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260620 11:41:56.959916 26031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:56.960217 26016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:56.960346 26016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "fd4d8a0299844fed9345cf5201d54aa9"
format_stamp: "Formatted at 2026-06-20 11:41:56 on dist-test-slave-szdr"
server_key: "b80411076ff6f8b6e86a5d794a726ce2"
server_key_iv: "7423bc4800f34c18d32a7fc3f49c1117"
server_key_version: "encryptionkey@0"
I20260620 11:41:56.960439 26016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:56.964561 26016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:56.967712 26016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:56.967919 26016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:56.972946 26016 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33287
I20260620 11:41:56.972976 26083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33287 every 8 connection(s)
I20260620 11:41:56.973392 26016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:56.974704 26084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:56.977345 26084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9: Bootstrap starting.
I20260620 11:41:56.978130 26084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:56.978459 26084 log.cc:824] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:56.979275 26084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9: No bootstrap required, opened a new log
I20260620 11:41:56.979329 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26016
I20260620 11:41:56.979508 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:56.979725 5610 external_mini_cluster.cc:1428] Setting key 922e3b2d45dcd29cc2407753605846c8
I20260620 11:41:56.981599 26084 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4d8a0299844fed9345cf5201d54aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33287 } }
I20260620 11:41:56.981843 26084 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:56.981936 26084 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4d8a0299844fed9345cf5201d54aa9, State: Initialized, Role: FOLLOWER
I20260620 11:41:56.982146 26084 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4d8a0299844fed9345cf5201d54aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33287 } }
I20260620 11:41:56.982252 26084 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:56.982278 26084 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:56.982405 26084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:56.983197 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:56.980321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39089 (local address 127.0.0.1:33287)
0620 11:41:56.980519 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:56.980531 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:56.980569 (+ 38us) server_negotiation.cc:407] Connection header received
0620 11:41:56.981584 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:56.981593 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:56.981629 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:56.981778 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:56.982639 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:56.982643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:56.982643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:56.982648 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:56.982689 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:56.982700 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:56.982765 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:56.982851 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 11:41:56.982945 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260620 11:41:56.983748 26084 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4d8a0299844fed9345cf5201d54aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33287 } }
I20260620 11:41:56.983896 26084 leader_election.cc:302] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd4d8a0299844fed9345cf5201d54aa9; no voters:
I20260620 11:41:56.984120 26084 leader_election.cc:288] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:56.984156 26089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:56.984201 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:41:56.984406 26084 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:56.984450 26089 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [term 1 LEADER]: Becoming Leader. State: Replica: fd4d8a0299844fed9345cf5201d54aa9, State: Running, Role: LEADER
I20260620 11:41:56.984594 26089 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4d8a0299844fed9345cf5201d54aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33287 } }
I20260620 11:41:56.985018 26090 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd4d8a0299844fed9345cf5201d54aa9. Latest consensus state: current_term: 1 leader_uuid: "fd4d8a0299844fed9345cf5201d54aa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4d8a0299844fed9345cf5201d54aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33287 } } }
I20260620 11:41:56.985131 26090 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:56.985018 26091 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd4d8a0299844fed9345cf5201d54aa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4d8a0299844fed9345cf5201d54aa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33287 } } }
I20260620 11:41:56.985270 26091 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:56.986157 26093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:56.986729 26093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:56.988760 26093 catalog_manager.cc:1347] Generated new cluster ID: 12f202785c5547b682b702bfb5f82855
I20260620 11:41:56.988843 26093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:56.988915 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:56.985950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40367 (local address 127.0.0.1:33287)
0620 11:41:56.986030 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:56.986032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:56.987561 (+ 1529us) server_negotiation.cc:407] Connection header received
0620 11:41:56.987596 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:56.987597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:56.987604 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:56.987676 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:56.988535 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:56.988536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:56.988537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:56.988541 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:56.988559 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:56.988563 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:56.988630 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:56.988799 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 11:41:56.988817 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 11:41:56.996662 26093 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:56.996876 26093 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:57.015007 26093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd4d8a0299844fed9345cf5201d54aa9: Generated new TSK 0
I20260620 11:41:57.015275 26093 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:57.023388 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26016
2026-06-20T11:41:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:57 dist-test-slave-szdr krb5kdc[26115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:57 dist-test-slave-szdr krb5kdc[26115](info): set up 2 sockets
Jun 20 11:41:57 dist-test-slave-szdr krb5kdc[26115](info): commencing operation
krb5kdc: starting...
W20260620 11:41:58.054236 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:58 dist-test-slave-szdr krb5kdc[26115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:41:58.198062 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45485
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:58.285774 26128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:58.286052 26128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:58.286080 26128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:58.287474 26128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:58.287529 26128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:58.287544 26128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:58.287557 26128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:58.287570 26128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:58.289157 26128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:58.289376 26128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:58.289630 26128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:58.294749 26136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:58.294842 26134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:58.294847 26133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:58.295491 26128 server_base.cc:1034] running on GCE node
I20260620 11:41:58.295778 26128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:58.296114 26128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:58.297291 26128 hybrid_clock.cc:648] HybridClock initialized: now 1781955718297281 us; error 53 us; skew 500 ppm
Jun 20 11:41:58 dist-test-slave-szdr krb5kdc[26115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955718, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:58.300209 26128 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:41:58.300719 26128 webserver.cc:460] Webserver started at http://127.0.0.1:38279/ using document root <none> and password file <none>
I20260620 11:41:58.300978 26128 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:58.301051 26128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:58.301216 26128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:58.302448 26128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "307d3518814040b9890c126d585ffd7f"
format_stamp: "Formatted at 2026-06-20 11:41:58 on dist-test-slave-szdr"
server_key: "c339ff71673c21533751664f09b66674"
server_key_iv: "7a3858f725c6bd007519da7387a57012"
server_key_version: "encryptionkey@0"
I20260620 11:41:58.302836 26128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "307d3518814040b9890c126d585ffd7f"
format_stamp: "Formatted at 2026-06-20 11:41:58 on dist-test-slave-szdr"
server_key: "c339ff71673c21533751664f09b66674"
server_key_iv: "7a3858f725c6bd007519da7387a57012"
server_key_version: "encryptionkey@0"
I20260620 11:41:58.304399 26128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:41:58.305317 26143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:58.305537 26128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:41:58.305649 26128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "307d3518814040b9890c126d585ffd7f"
format_stamp: "Formatted at 2026-06-20 11:41:58 on dist-test-slave-szdr"
server_key: "c339ff71673c21533751664f09b66674"
server_key_iv: "7a3858f725c6bd007519da7387a57012"
server_key_version: "encryptionkey@0"
I20260620 11:41:58.305727 26128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:58.324509 26128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:58.327178 26128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:58.327356 26128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:58.331375 26128 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45485
I20260620 11:41:58.331419 26195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45485 every 8 connection(s)
I20260620 11:41:58.331840 26128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:58.332901 26196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:58.333561 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26128
I20260620 11:41:58.333696 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:58.333891 5610 external_mini_cluster.cc:1428] Setting key e913d55b4d160b791d7b4c65239c4c5e
I20260620 11:41:58.336123 26196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f: Bootstrap starting.
I20260620 11:41:58.336750 26196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:58.337060 26196 log.cc:824] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:58.337953 26196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f: No bootstrap required, opened a new log
I20260620 11:41:58.340509 26196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "307d3518814040b9890c126d585ffd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45485 } }
I20260620 11:41:58.340695 26196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:58.340741 26196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 307d3518814040b9890c126d585ffd7f, State: Initialized, Role: FOLLOWER
I20260620 11:41:58.340889 26196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "307d3518814040b9890c126d585ffd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45485 } }
I20260620 11:41:58.340976 26196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:58.341008 26196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Jun 20 11:41:58 dist-test-slave-szdr krb5kdc[26115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 11:41:58.341060 26196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:58.341811 26196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "307d3518814040b9890c126d585ffd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45485 } }
I20260620 11:41:58.341964 26196 leader_election.cc:302] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 307d3518814040b9890c126d585ffd7f; no voters:
I20260620 11:41:58.342211 26196 leader_election.cc:288] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:58.342262 26201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:58.342458 26201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [term 1 LEADER]: Becoming Leader. State: Replica: 307d3518814040b9890c126d585ffd7f, State: Running, Role: LEADER
I20260620 11:41:58.342545 26196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:58.342595 26201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "307d3518814040b9890c126d585ffd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45485 } }
I20260620 11:41:58.342979 26202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "307d3518814040b9890c126d585ffd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "307d3518814040b9890c126d585ffd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45485 } } }
I20260620 11:41:58.343092 26202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:58.342979 26203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 307d3518814040b9890c126d585ffd7f. Latest consensus state: current_term: 1 leader_uuid: "307d3518814040b9890c126d585ffd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "307d3518814040b9890c126d585ffd7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45485 } } }
I20260620 11:41:58.343394 26203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:58.343786 26208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:58.344430 26208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:58.345889 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:58.334480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50531 (local address 127.0.0.1:45485)
0620 11:41:58.334654 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:58.334663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:58.334689 (+ 26us) server_negotiation.cc:407] Connection header received
0620 11:41:58.335118 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:58.335125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:58.335171 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:58.335290 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:58.336116 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:58.337167 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:58.338279 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:58.338897 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:58.339012 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:58.341577 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:58.341582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:58.341582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:58.341596 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:58.344096 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:58.344548 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:58.344552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:58.344552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:58.344674 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:58.344903 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:58.344904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:58.344905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:58.345227 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:58.345378 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:58.345526 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 11:41:58.345677 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260620 11:41:58.346391 26208 catalog_manager.cc:1347] Generated new cluster ID: 7980ebd08c084b0c82110358c2940ece
I20260620 11:41:58.346442 26208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:58.347076 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:41:58.351727 26208 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:58.352288 26208 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:58.356938 26208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 307d3518814040b9890c126d585ffd7f: Generated new TSK 0
I20260620 11:41:58.357167 26208 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:58.358243 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:58.348713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37809 (local address 127.0.0.1:45485)
0620 11:41:58.348751 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:58.348753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:58.350145 (+ 1392us) server_negotiation.cc:407] Connection header received
0620 11:41:58.350253 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:58.350254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:58.350265 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:58.350318 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:58.351093 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:58.351742 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:58.352794 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:58.353127 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:58.353195 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:58.354810 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:58.354812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:58.354812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:41:58.354820 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:58.356503 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:58.357389 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:58.357390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:58.357391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:58.357456 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:41:58.357674 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:41:58.357675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:41:58.357676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:41:58.357793 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:58.357900 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:58.358044 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 11:41:58.358145 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:41:58.360440 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26128
2026-06-20T11:41:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:58 dist-test-slave-szdr krb5kdc[26227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:58 dist-test-slave-szdr krb5kdc[26227](info): set up 2 sockets
Jun 20 11:41:58 dist-test-slave-szdr krb5kdc[26227](info): commencing operation
krb5kdc: starting...
W20260620 11:41:59.398192 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:41:59 dist-test-slave-szdr krb5kdc[26227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:41:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:41:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:41:59.532923 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:41:59.618737 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:41:59.619037 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:41:59.619083 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:41:59.620405 26240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:41:59.620491 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:41:59.620515 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:41:59.620534 26240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:41:59.620564 26240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:41:59.622320 26240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:41:59.622654 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:41:59.622913 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:41:59.628263 26245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:59.628325 26246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:41:59.628263 26248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:41:59.628674 26240 server_base.cc:1034] running on GCE node
I20260620 11:41:59.628958 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:41:59.629330 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:41:59.630509 26240 hybrid_clock.cc:648] HybridClock initialized: now 1781955719630485 us; error 42 us; skew 500 ppm
Jun 20 11:41:59 dist-test-slave-szdr krb5kdc[26227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955719, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:41:59.639698 26240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:41:59.640184 26240 webserver.cc:460] Webserver started at http://127.0.0.1:33079/ using document root <none> and password file <none>
I20260620 11:41:59.640401 26240 fs_manager.cc:359] Metadata directory not provided
I20260620 11:41:59.640447 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:41:59.640556 26240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:41:59.641705 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "e38f034fcd6e4a6080fe50ed90356c67"
format_stamp: "Formatted at 2026-06-20 11:41:59 on dist-test-slave-szdr"
server_key: "890112b27bb641e3c74b882d0a7973b8"
server_key_iv: "eb5db4ef3fc6adcff01f792337a3dbd4"
server_key_version: "encryptionkey@0"
I20260620 11:41:59.642155 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "e38f034fcd6e4a6080fe50ed90356c67"
format_stamp: "Formatted at 2026-06-20 11:41:59 on dist-test-slave-szdr"
server_key: "890112b27bb641e3c74b882d0a7973b8"
server_key_iv: "eb5db4ef3fc6adcff01f792337a3dbd4"
server_key_version: "encryptionkey@0"
I20260620 11:41:59.643630 26240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:59.644469 26255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:41:59.644671 26240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:41:59.644797 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "e38f034fcd6e4a6080fe50ed90356c67"
format_stamp: "Formatted at 2026-06-20 11:41:59 on dist-test-slave-szdr"
server_key: "890112b27bb641e3c74b882d0a7973b8"
server_key_iv: "eb5db4ef3fc6adcff01f792337a3dbd4"
server_key_version: "encryptionkey@0"
I20260620 11:41:59.644886 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:41:59.659947 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:41:59.662774 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:41:59.662977 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:41:59.667376 26240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41401
I20260620 11:41:59.667426 26307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41401 every 8 connection(s)
I20260620 11:41:59.667819 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:41:59.668459 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26240
I20260620 11:41:59.668591 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:41:59.668802 5610 external_mini_cluster.cc:1428] Setting key a32b3898519c6bc9ed61a20720535992
I20260620 11:41:59.669108 26308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:41:59.672426 26308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67: Bootstrap starting.
I20260620 11:41:59.673147 26308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67: Neither blocks nor log segments found. Creating new log.
I20260620 11:41:59.673480 26308 log.cc:824] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67: Log is configured to *not* fsync() on all Append() calls
I20260620 11:41:59.674304 26308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67: No bootstrap required, opened a new log
I20260620 11:41:59.675251 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:59.669428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47355 (local address 127.0.0.1:41401)
0620 11:41:59.669648 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:59.669657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:59.669686 (+ 29us) server_negotiation.cc:407] Connection header received
0620 11:41:59.670425 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:59.670434 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:59.670493 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:59.670681 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:59.670958 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:59.671959 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:59.672959 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:59.673483 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:59.673625 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:59.674542 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:59.674548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:59.674548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:59.674739 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:59.674776 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:59.674788 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:59.674872 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:59.674954 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 11:41:59.675031 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":96,"threads_started":1}
I20260620 11:41:59.676417 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:41:59.676779 26308 raft_consensus.cc:348] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38f034fcd6e4a6080fe50ed90356c67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41401 } }
I20260620 11:41:59.676963 26308 raft_consensus.cc:374] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:41:59.677012 26308 raft_consensus.cc:729] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38f034fcd6e4a6080fe50ed90356c67, State: Initialized, Role: FOLLOWER
I20260620 11:41:59.677153 26308 consensus_queue.cc:260] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38f034fcd6e4a6080fe50ed90356c67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41401 } }
I20260620 11:41:59.677253 26308 raft_consensus.cc:388] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:41:59.677297 26308 raft_consensus.cc:482] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:41:59.677349 26308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:41:59.678161 26308 raft_consensus.cc:504] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38f034fcd6e4a6080fe50ed90356c67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41401 } }
I20260620 11:41:59.678325 26308 leader_election.cc:302] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e38f034fcd6e4a6080fe50ed90356c67; no voters:
I20260620 11:41:59.678535 26308 leader_election.cc:288] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:41:59.678565 26318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:41:59.678874 26318 raft_consensus.cc:686] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [term 1 LEADER]: Becoming Leader. State: Replica: e38f034fcd6e4a6080fe50ed90356c67, State: Running, Role: LEADER
I20260620 11:41:59.678887 26308 sys_catalog.cc:564] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:41:59.679647 26318 consensus_queue.cc:237] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38f034fcd6e4a6080fe50ed90356c67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41401 } }
I20260620 11:41:59.680481 26319 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e38f034fcd6e4a6080fe50ed90356c67. Latest consensus state: current_term: 1 leader_uuid: "e38f034fcd6e4a6080fe50ed90356c67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38f034fcd6e4a6080fe50ed90356c67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41401 } } }
I20260620 11:41:59.680624 26320 sys_catalog.cc:455] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e38f034fcd6e4a6080fe50ed90356c67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e38f034fcd6e4a6080fe50ed90356c67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41401 } } }
I20260620 11:41:59.681182 26319 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [sys.catalog]: This master's current role is: LEADER
I20260620 11:41:59.681193 26320 sys_catalog.cc:458] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67 [sys.catalog]: This master's current role is: LEADER
W20260620 11:41:59.681430 26332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:41:59.681493 26332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:41:59.681770 26336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:41:59.682020 26336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:41:59.682165 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:41:59.678202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34157 (local address 127.0.0.1:41401)
0620 11:41:59.678275 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 11:41:59.678277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:41:59.678407 (+ 130us) server_negotiation.cc:407] Connection header received
0620 11:41:59.678503 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:41:59.678504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:41:59.678516 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:41:59.678568 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:41:59.678729 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:59.679374 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:59.680524 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:41:59.680878 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:41:59.680931 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:41:59.681850 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:41:59.681852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:41:59.681852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:41:59.681858 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:41:59.681873 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:41:59.681878 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:41:59.681907 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0620 11:41:59.682063 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 11:41:59.682091 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 11:41:59.683683 26336 catalog_manager.cc:1347] Generated new cluster ID: b692c0025fc94cec8658d964033a9572
I20260620 11:41:59.683738 26336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:41:59.703745 26336 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:41:59.704604 26336 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:41:59.708891 26336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e38f034fcd6e4a6080fe50ed90356c67: Generated new TSK 0
I20260620 11:41:59.709134 26336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:41:59.712353 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26240
2026-06-20T11:41:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:41:59 dist-test-slave-szdr krb5kdc[26340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:41:59 dist-test-slave-szdr krb5kdc[26340](info): set up 2 sockets
Jun 20 11:41:59 dist-test-slave-szdr krb5kdc[26340](info): commencing operation
krb5kdc: starting...
W20260620 11:42:00.754495 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:00 dist-test-slave-szdr krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:42:00.889932 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:00.981750 26353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:00.982071 26353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:00.982103 26353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:00.983520 26353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:00.983573 26353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:00.983589 26353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:00.983604 26353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:00.983618 26353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:00.985383 26353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:00.985757 26353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:00.986117 26353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:00.992362 26359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:00.992543 26353 server_base.cc:1034] running on GCE node
W20260620 11:42:00.992333 26358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:00.992333 26361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:00.992919 26353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:00.993297 26353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:00.994491 26353 hybrid_clock.cc:648] HybridClock initialized: now 1781955720994466 us; error 33 us; skew 500 ppm
Jun 20 11:42:00 dist-test-slave-szdr krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955720, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:00.997363 26353 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:42:00.997815 26353 webserver.cc:460] Webserver started at http://127.0.0.1:44321/ using document root <none> and password file <none>
I20260620 11:42:00.998111 26353 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:00.998162 26353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:00.998266 26353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:00.999379 26353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "d035b093188549f4b4c96cc00c4b692d"
format_stamp: "Formatted at 2026-06-20 11:42:00 on dist-test-slave-szdr"
server_key: "75ace7a68a4ea3c874bcfe6f88d8ff95"
server_key_iv: "72ef4e9712b3319106bcb4f5d04e7355"
server_key_version: "encryptionkey@0"
I20260620 11:42:00.999711 26353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "d035b093188549f4b4c96cc00c4b692d"
format_stamp: "Formatted at 2026-06-20 11:42:00 on dist-test-slave-szdr"
server_key: "75ace7a68a4ea3c874bcfe6f88d8ff95"
server_key_iv: "72ef4e9712b3319106bcb4f5d04e7355"
server_key_version: "encryptionkey@0"
I20260620 11:42:01.001134 26353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:01.002064 26368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:01.002233 26353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:01.002328 26353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "d035b093188549f4b4c96cc00c4b692d"
format_stamp: "Formatted at 2026-06-20 11:42:00 on dist-test-slave-szdr"
server_key: "75ace7a68a4ea3c874bcfe6f88d8ff95"
server_key_iv: "72ef4e9712b3319106bcb4f5d04e7355"
server_key_version: "encryptionkey@0"
I20260620 11:42:01.002401 26353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:01.005919 26353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:01.008576 26353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:01.008703 26353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:01.012820 26353 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43675
I20260620 11:42:01.012985 26420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43675 every 8 connection(s)
I20260620 11:42:01.013221 26353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:01.014394 26421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:01.016609 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26353
I20260620 11:42:01.016736 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:01.016949 5610 external_mini_cluster.cc:1428] Setting key 5f86cd8ca06489e25e96d445a2f2d5bf
I20260620 11:42:01.017398 26421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d: Bootstrap starting.
I20260620 11:42:01.018224 26421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:01.018559 26421 log.cc:824] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:01.019462 26421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d: No bootstrap required, opened a new log
I20260620 11:42:01.019948 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:01.017561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56955 (local address 127.0.0.1:43675)
0620 11:42:01.017773 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:01.017784 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:01.017822 (+ 38us) server_negotiation.cc:407] Connection header received
0620 11:42:01.018433 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:01.018442 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:01.018474 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:01.018631 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:01.019417 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:01.019421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:01.019421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:01.019425 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:01.019466 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:01.019476 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:01.019597 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:01.019675 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 11:42:01.019730 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260620 11:42:01.021088 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:01.022006 26421 raft_consensus.cc:348] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d035b093188549f4b4c96cc00c4b692d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43675 } }
I20260620 11:42:01.022218 26421 raft_consensus.cc:374] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:01.022261 26421 raft_consensus.cc:729] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d035b093188549f4b4c96cc00c4b692d, State: Initialized, Role: FOLLOWER
I20260620 11:42:01.022414 26421 consensus_queue.cc:260] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d035b093188549f4b4c96cc00c4b692d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43675 } }
I20260620 11:42:01.022528 26421 raft_consensus.cc:388] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:01.022573 26421 raft_consensus.cc:482] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:01.022629 26421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:01.023337 26421 raft_consensus.cc:504] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d035b093188549f4b4c96cc00c4b692d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43675 } }
I20260620 11:42:01.023469 26421 leader_election.cc:302] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d035b093188549f4b4c96cc00c4b692d; no voters:
I20260620 11:42:01.023658 26421 leader_election.cc:288] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:01.023698 26431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:01.023947 26421 sys_catalog.cc:564] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:01.024001 26431 raft_consensus.cc:686] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [term 1 LEADER]: Becoming Leader. State: Replica: d035b093188549f4b4c96cc00c4b692d, State: Running, Role: LEADER
I20260620 11:42:01.024241 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:01.022877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46457 (local address 127.0.0.1:43675)
0620 11:42:01.022938 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:01.022940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:01.023057 (+ 117us) server_negotiation.cc:407] Connection header received
0620 11:42:01.023134 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:01.023135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:01.023140 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:01.023197 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:01.023878 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:01.023879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:01.023879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:01.023884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:01.023899 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:01.023903 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:01.023969 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:01.024072 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 11:42:01.024087 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 11:42:01.024088 26431 consensus_queue.cc:237] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d035b093188549f4b4c96cc00c4b692d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43675 } }
I20260620 11:42:01.024801 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d035b093188549f4b4c96cc00c4b692d. Latest consensus state: current_term: 1 leader_uuid: "d035b093188549f4b4c96cc00c4b692d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d035b093188549f4b4c96cc00c4b692d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43675 } } }
I20260620 11:42:01.024883 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:01.025036 26432 sys_catalog.cc:455] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d035b093188549f4b4c96cc00c4b692d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d035b093188549f4b4c96cc00c4b692d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43675 } } }
I20260620 11:42:01.025120 26432 sys_catalog.cc:458] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:01.025141 26440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:01.025595 26440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:01.027120 26448 mvcc.cc:205] Tried to move back new op lower bound from 7298890633325334528 to 7298890633316450304. Current Snapshot: MvccSnapshot[applied={T|T < 7298890633325334528}]
I20260620 11:42:01.027709 26440 catalog_manager.cc:1347] Generated new cluster ID: e8efffc605234e8986f2e1eb6dec83b6
I20260620 11:42:01.027760 26440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:01.036116 26440 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:01.036378 26440 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:01.046267 26440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d035b093188549f4b4c96cc00c4b692d: Generated new TSK 0
I20260620 11:42:01.046552 26440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:01.051245 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26353
2026-06-20T11:42:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:01 dist-test-slave-szdr krb5kdc[26452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:01 dist-test-slave-szdr krb5kdc[26452](info): set up 2 sockets
Jun 20 11:42:01 dist-test-slave-szdr krb5kdc[26452](info): commencing operation
krb5kdc: starting...
W20260620 11:42:02.102171 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:02 dist-test-slave-szdr krb5kdc[26452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:42:02.233739 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:02.320636 26465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:02.320978 26465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:02.321022 26465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:02.322402 26465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:02.322477 26465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:02.322500 26465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:02.322521 26465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:02.322553 26465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:02.324203 26465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:02.324536 26465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:02.324802 26465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:02.330183 26470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:02.330207 26471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:02.330236 26465 server_base.cc:1034] running on GCE node
W20260620 11:42:02.330205 26473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:02.330649 26465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:02.330971 26465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:02.332142 26465 hybrid_clock.cc:648] HybridClock initialized: now 1781955722332116 us; error 48 us; skew 500 ppm
Jun 20 11:42:02 dist-test-slave-szdr krb5kdc[26452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955722, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:02.334978 26465 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:42:02.335556 26465 webserver.cc:460] Webserver started at http://127.0.0.1:46109/ using document root <none> and password file <none>
I20260620 11:42:02.335819 26465 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:02.335892 26465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:02.336050 26465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:02.337173 26465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "ddab967f43eb403084984a363c188b2d"
format_stamp: "Formatted at 2026-06-20 11:42:02 on dist-test-slave-szdr"
server_key: "d8f0ea778157869ce69f8de6dabea077"
server_key_iv: "2e67f39d6813dad5858d2b9d9149e175"
server_key_version: "encryptionkey@0"
I20260620 11:42:02.337565 26465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "ddab967f43eb403084984a363c188b2d"
format_stamp: "Formatted at 2026-06-20 11:42:02 on dist-test-slave-szdr"
server_key: "d8f0ea778157869ce69f8de6dabea077"
server_key_iv: "2e67f39d6813dad5858d2b9d9149e175"
server_key_version: "encryptionkey@0"
I20260620 11:42:02.339109 26465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:42:02.339970 26480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:02.340162 26465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:42:02.340277 26465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "ddab967f43eb403084984a363c188b2d"
format_stamp: "Formatted at 2026-06-20 11:42:02 on dist-test-slave-szdr"
server_key: "d8f0ea778157869ce69f8de6dabea077"
server_key_iv: "2e67f39d6813dad5858d2b9d9149e175"
server_key_version: "encryptionkey@0"
I20260620 11:42:02.340376 26465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:02.351748 26465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:02.354538 26465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:02.354732 26465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:02.359156 26465 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34745
I20260620 11:42:02.359189 26532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34745 every 8 connection(s)
I20260620 11:42:02.359602 26465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:02.360682 26533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:02.363238 26533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d: Bootstrap starting.
I20260620 11:42:02.363914 26533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:02.364228 26533 log.cc:824] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:02.365051 26533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d: No bootstrap required, opened a new log
I20260620 11:42:02.366797 26533 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddab967f43eb403084984a363c188b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34745 } }
I20260620 11:42:02.366959 26533 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:02.366984 26533 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddab967f43eb403084984a363c188b2d, State: Initialized, Role: FOLLOWER
I20260620 11:42:02.367168 26533 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddab967f43eb403084984a363c188b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34745 } }
I20260620 11:42:02.367234 26533 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:02.367280 26533 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:02.367342 26533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:02.367946 26533 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddab967f43eb403084984a363c188b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34745 } }
I20260620 11:42:02.368076 26533 leader_election.cc:302] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddab967f43eb403084984a363c188b2d; no voters:
I20260620 11:42:02.368284 26533 leader_election.cc:288] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:02.368338 26536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:02.368558 26536 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [term 1 LEADER]: Becoming Leader. State: Replica: ddab967f43eb403084984a363c188b2d, State: Running, Role: LEADER
I20260620 11:42:02.368615 26533 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:02.368656 26536 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddab967f43eb403084984a363c188b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34745 } }
I20260620 11:42:02.369374 26537 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddab967f43eb403084984a363c188b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddab967f43eb403084984a363c188b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34745 } } }
I20260620 11:42:02.369498 26537 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:02.369690 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26465
I20260620 11:42:02.369827 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:02.369874 26538 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddab967f43eb403084984a363c188b2d. Latest consensus state: current_term: 1 leader_uuid: "ddab967f43eb403084984a363c188b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddab967f43eb403084984a363c188b2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34745 } } }
I20260620 11:42:02.369940 26538 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:02.370054 5610 external_mini_cluster.cc:1428] Setting key f2dac05dab7dacb6ccb5a7ccf0948a5d
I20260620 11:42:02.370280 26545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:02.371075 26545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:02.372697 26545 catalog_manager.cc:1347] Generated new cluster ID: b6819e7dd41040959285086bf7f2d2fa
I20260620 11:42:02.372753 26545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:42:02 dist-test-slave-szdr krb5kdc[26452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 11:42:02.378870 26545 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:02.379611 26545 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:02.381319 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:02.370595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:40185 (local address 127.0.0.1:34745)
0620 11:42:02.370762 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:02.370775 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:02.370816 (+ 41us) server_negotiation.cc:407] Connection header received
0620 11:42:02.370935 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:02.370941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:02.370995 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:02.371146 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:02.371843 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:02.373039 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:02.373840 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:02.374386 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:02.374510 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:02.377047 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:02.377052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:02.377053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:02.377067 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:02.379602 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:02.380050 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:02.380060 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:02.380061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:02.380156 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:02.380401 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:02.380403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:02.380403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:02.380645 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:02.380863 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:02.380959 (+ 96us) server_negotiation.cc:299] Negotiation successful
0620 11:42:02.381075 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260620 11:42:02.382439 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:02.389946 26545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddab967f43eb403084984a363c188b2d: Generated new TSK 0
I20260620 11:42:02.390311 26545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:02.390890 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:02.383939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:43129 (local address 127.0.0.1:34745)
0620 11:42:02.383982 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:02.383984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:02.384112 (+ 128us) server_negotiation.cc:407] Connection header received
0620 11:42:02.384188 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:02.384189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:02.384199 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:02.384255 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:02.384796 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:02.385359 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:02.386218 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:02.386462 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:02.386544 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:02.387912 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:02.387913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:02.387914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:02.387922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:02.389545 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:02.390054 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:02.390055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:02.390055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:02.390124 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:02.390360 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:02.390361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:02.390361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:02.390481 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:02.390594 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:02.390722 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 11:42:02.390790 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:42:02.393107 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26465
W20260620 11:42:02.397176 26450 connection.cc:535] client connection to 127.0.0.1:34745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34745): Connection reset by peer (error 104)
2026-06-20T11:42:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:02 dist-test-slave-szdr krb5kdc[26564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:02 dist-test-slave-szdr krb5kdc[26564](info): set up 2 sockets
Jun 20 11:42:02 dist-test-slave-szdr krb5kdc[26564](info): commencing operation
krb5kdc: starting...
W20260620 11:42:03.454308 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:03 dist-test-slave-szdr krb5kdc[26564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:42:03.581776 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:03.667848 26577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:03.668133 26577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:03.668171 26577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:03.669425 26577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:03.669512 26577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:03.669536 26577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:03.669559 26577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:03.669587 26577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:03.671293 26577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:03.671602 26577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:03.671859 26577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 11:42:03.677237 26577 server_base.cc:1034] running on GCE node
W20260620 11:42:03.677202 26582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:03.677213 26583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:03.677182 26585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:03.677677 26577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:03.677950 26577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:03.679159 26577 hybrid_clock.cc:648] HybridClock initialized: now 1781955723679138 us; error 39 us; skew 500 ppm
Jun 20 11:42:03 dist-test-slave-szdr krb5kdc[26564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955723, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:03.682067 26577 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:42:03.682583 26577 webserver.cc:460] Webserver started at http://127.0.0.1:42079/ using document root <none> and password file <none>
I20260620 11:42:03.682847 26577 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:03.682919 26577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:03.683070 26577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:03.684175 26577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "835280eb0c3049fdba755e8aac208015"
format_stamp: "Formatted at 2026-06-20 11:42:03 on dist-test-slave-szdr"
server_key: "0a12fdcacc04522b89060cab1ad75952"
server_key_iv: "a7f7f2679d7d7604a8bf20662e0d247e"
server_key_version: "encryptionkey@0"
I20260620 11:42:03.684552 26577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "835280eb0c3049fdba755e8aac208015"
format_stamp: "Formatted at 2026-06-20 11:42:03 on dist-test-slave-szdr"
server_key: "0a12fdcacc04522b89060cab1ad75952"
server_key_iv: "a7f7f2679d7d7604a8bf20662e0d247e"
server_key_version: "encryptionkey@0"
I20260620 11:42:03.685999 26577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:42:03.686882 26592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:03.687084 26577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:03.687234 26577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "835280eb0c3049fdba755e8aac208015"
format_stamp: "Formatted at 2026-06-20 11:42:03 on dist-test-slave-szdr"
server_key: "0a12fdcacc04522b89060cab1ad75952"
server_key_iv: "a7f7f2679d7d7604a8bf20662e0d247e"
server_key_version: "encryptionkey@0"
I20260620 11:42:03.687305 26577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:03.702360 26577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:03.704932 26577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:03.705114 26577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:03.709278 26577 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44837
I20260620 11:42:03.709369 26644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44837 every 8 connection(s)
I20260620 11:42:03.709735 26577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:03.710865 26645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:03.713438 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015: Bootstrap starting.
I20260620 11:42:03.714139 26645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:03.714468 26645 log.cc:824] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:03.715270 26645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015: No bootstrap required, opened a new log
I20260620 11:42:03.716996 26645 raft_consensus.cc:348] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835280eb0c3049fdba755e8aac208015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44837 } }
I20260620 11:42:03.717149 26645 raft_consensus.cc:374] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:03.717218 26645 raft_consensus.cc:729] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 835280eb0c3049fdba755e8aac208015, State: Initialized, Role: FOLLOWER
I20260620 11:42:03.717372 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26577
I20260620 11:42:03.717401 26645 consensus_queue.cc:260] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835280eb0c3049fdba755e8aac208015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44837 } }
I20260620 11:42:03.717496 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:03.717502 26645 raft_consensus.cc:388] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:03.717537 26645 raft_consensus.cc:482] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:03.717594 26645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:03.717691 5610 external_mini_cluster.cc:1428] Setting key 2038d7e0e62e7801a32c268130fd7378
I20260620 11:42:03.718461 26645 raft_consensus.cc:504] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835280eb0c3049fdba755e8aac208015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44837 } }
I20260620 11:42:03.718614 26645 leader_election.cc:302] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 835280eb0c3049fdba755e8aac208015; no voters:
I20260620 11:42:03.718812 26645 leader_election.cc:288] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:03.718847 26650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:03.719069 26650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [term 1 LEADER]: Becoming Leader. State: Replica: 835280eb0c3049fdba755e8aac208015, State: Running, Role: LEADER
I20260620 11:42:03.719100 26645 sys_catalog.cc:564] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:03.719233 26650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835280eb0c3049fdba755e8aac208015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44837 } }
I20260620 11:42:03.720683 26652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "835280eb0c3049fdba755e8aac208015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835280eb0c3049fdba755e8aac208015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44837 } } }
I20260620 11:42:03.720876 26652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [sys.catalog]: This master's current role is: LEADER
W20260620 11:42:03.721045 26664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:42:03.721112 26664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:42:03.721179 26666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:03.721325 26651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 835280eb0c3049fdba755e8aac208015. Latest consensus state: current_term: 1 leader_uuid: "835280eb0c3049fdba755e8aac208015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "835280eb0c3049fdba755e8aac208015" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44837 } } }
I20260620 11:42:03.721367 26666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:03.721412 26651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:03.724370 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:03.718601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:52169 (local address 127.0.0.1:44837)
0620 11:42:03.718837 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:03.718848 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:03.718890 (+ 42us) server_negotiation.cc:407] Connection header received
0620 11:42:03.719032 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:03.719040 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:03.719136 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:03.719317 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:03.719555 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:03.721122 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:03.722119 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:03.722642 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:03.722788 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:03.723567 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:03.723571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:03.723572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:03.723806 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:03.723849 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:03.723861 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:03.723935 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:03.724061 (+ 126us) server_negotiation.cc:299] Negotiation successful
0620 11:42:03.724123 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
I20260620 11:42:03.725620 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:03.730508 26666 catalog_manager.cc:1347] Generated new cluster ID: bebb886db03b458cbbe4ae6960075fa0
I20260620 11:42:03.730599 26666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:03.733263 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:03.727185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:38783 (local address 127.0.0.1:44837)
0620 11:42:03.727231 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:03.727234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:03.730273 (+ 3039us) server_negotiation.cc:407] Connection header received
0620 11:42:03.730307 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:03.730308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:03.730322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:03.730395 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:03.730559 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:03.731110 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:03.731881 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:03.732133 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:03.732227 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:03.732969 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:03.732970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:03.732971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:03.732977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:03.732994 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:03.732998 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:03.733051 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:03.733133 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 11:42:03.733154 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:42:03.747423 26666 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:03.748174 26666 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:03.756999 26666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 835280eb0c3049fdba755e8aac208015: Generated new TSK 0
I20260620 11:42:03.757246 26666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:03.764685 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26577
W20260620 11:42:03.768911 26562 connection.cc:535] client connection to 127.0.0.1:44837 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44837): Connection reset by peer (error 104)
2026-06-20T11:42:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1371 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:03 dist-test-slave-szdr krb5kdc[26676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:03 dist-test-slave-szdr krb5kdc[26676](info): set up 2 sockets
Jun 20 11:42:03 dist-test-slave-szdr krb5kdc[26676](info): commencing operation
krb5kdc: starting...
W20260620 11:42:04.798381 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:04 dist-test-slave-szdr krb5kdc[26676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:42:04.925597 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:05.011539 26689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:05.011785 26689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:05.011811 26689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:05.013115 26689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:05.013159 26689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:05.013175 26689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:05.013187 26689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:05.013199 26689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:05.014917 26689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:05.015166 26689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:05.015362 26689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:05.020835 26694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:05.021152 26689 server_base.cc:1034] running on GCE node
W20260620 11:42:05.020865 26695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:05.020897 26697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:05.021574 26689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:05.022001 26689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:05.023200 26689 hybrid_clock.cc:648] HybridClock initialized: now 1781955725023196 us; error 38 us; skew 500 ppm
Jun 20 11:42:05 dist-test-slave-szdr krb5kdc[26676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:05.029177 26689 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:42:05.029713 26689 webserver.cc:460] Webserver started at http://127.0.0.1:44445/ using document root <none> and password file <none>
I20260620 11:42:05.030071 26689 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:05.030151 26689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:05.030319 26689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:05.031504 26689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "8062e93a08294b148534c0e3219866a7"
format_stamp: "Formatted at 2026-06-20 11:42:05 on dist-test-slave-szdr"
server_key: "b6fb3cfa0821a001adcc06a9d67475e5"
server_key_iv: "325871e2820376e7c517a54a52042126"
server_key_version: "encryptionkey@0"
I20260620 11:42:05.031908 26689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "8062e93a08294b148534c0e3219866a7"
format_stamp: "Formatted at 2026-06-20 11:42:05 on dist-test-slave-szdr"
server_key: "b6fb3cfa0821a001adcc06a9d67475e5"
server_key_iv: "325871e2820376e7c517a54a52042126"
server_key_version: "encryptionkey@0"
I20260620 11:42:05.033389 26689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:05.034281 26704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:05.034543 26689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:05.034658 26689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "8062e93a08294b148534c0e3219866a7"
format_stamp: "Formatted at 2026-06-20 11:42:05 on dist-test-slave-szdr"
server_key: "b6fb3cfa0821a001adcc06a9d67475e5"
server_key_iv: "325871e2820376e7c517a54a52042126"
server_key_version: "encryptionkey@0"
I20260620 11:42:05.034741 26689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:05.038343 26689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:05.040759 26689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:05.040938 26689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:05.045300 26689 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42457
I20260620 11:42:05.045337 26756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42457 every 8 connection(s)
I20260620 11:42:05.045708 26689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:05.046914 26757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:05.049372 26757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7: Bootstrap starting.
I20260620 11:42:05.050082 26757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:05.050359 26757 log.cc:824] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:05.051113 26757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7: No bootstrap required, opened a new log
I20260620 11:42:05.051216 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26689
I20260620 11:42:05.051367 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:05.051560 5610 external_mini_cluster.cc:1428] Setting key 9cd116d0220b8a2b87e62c83fc5e5fcf
I20260620 11:42:05.053349 26757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062e93a08294b148534c0e3219866a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42457 } }
I20260620 11:42:05.053577 26757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:05.053627 26757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8062e93a08294b148534c0e3219866a7, State: Initialized, Role: FOLLOWER
I20260620 11:42:05.053833 26757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062e93a08294b148534c0e3219866a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42457 } }
I20260620 11:42:05.053968 26757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:05.054059 26757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:05.054117 26757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:05.054847 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:05.052150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:35419 (local address 127.0.0.1:42457)
0620 11:42:05.052338 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:05.052347 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:05.052373 (+ 26us) server_negotiation.cc:407] Connection header received
0620 11:42:05.053318 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:05.053326 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:05.053359 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:05.053496 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:05.054338 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:05.054341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:05.054342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:05.054346 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:05.054386 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:05.054398 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:05.054450 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:05.054549 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 11:42:05.054611 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260620 11:42:05.055456 26757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062e93a08294b148534c0e3219866a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42457 } }
I20260620 11:42:05.055585 26757 leader_election.cc:302] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8062e93a08294b148534c0e3219866a7; no voters:
I20260620 11:42:05.055765 26757 leader_election.cc:288] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:05.055819 26762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:05.055968 26762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [term 1 LEADER]: Becoming Leader. State: Replica: 8062e93a08294b148534c0e3219866a7, State: Running, Role: LEADER
I20260620 11:42:05.055979 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:05.056052 26757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:05.056107 26762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062e93a08294b148534c0e3219866a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42457 } }
I20260620 11:42:05.056443 26762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8062e93a08294b148534c0e3219866a7. Latest consensus state: current_term: 1 leader_uuid: "8062e93a08294b148534c0e3219866a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062e93a08294b148534c0e3219866a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42457 } } }
I20260620 11:42:05.056456 26763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8062e93a08294b148534c0e3219866a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8062e93a08294b148534c0e3219866a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42457 } } }
I20260620 11:42:05.056533 26762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:05.056567 26763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:05.056756 26769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:05.057296 26769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:05.058943 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:05.057681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:57913 (local address 127.0.0.1:42457)
0620 11:42:05.057716 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:05.057718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:05.057797 (+ 79us) server_negotiation.cc:407] Connection header received
0620 11:42:05.057851 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:05.057852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:05.057857 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:05.057918 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:05.058729 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:05.058731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:05.058731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:05.058736 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:05.058752 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:05.058756 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:05.058809 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:05.058861 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 11:42:05.058877 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 11:42:05.059623 26769 catalog_manager.cc:1347] Generated new cluster ID: dd0b804d99864e3b92179d8ec3075bc3
I20260620 11:42:05.059674 26769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:05.069725 26769 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:05.069952 26769 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:05.075620 26769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8062e93a08294b148534c0e3219866a7: Generated new TSK 0
I20260620 11:42:05.075881 26769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:05.081640 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26689
2026-06-20T11:42:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:05 dist-test-slave-szdr krb5kdc[26788](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:05 dist-test-slave-szdr krb5kdc[26788](info): set up 2 sockets
Jun 20 11:42:05 dist-test-slave-szdr krb5kdc[26788](info): commencing operation
krb5kdc: starting...
W20260620 11:42:06.118697 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:06 dist-test-slave-szdr krb5kdc[26788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:42:06.249356 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:06.333102 26801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:06.333328 26801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:06.333352 26801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:06.334714 26801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:06.334762 26801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:06.334777 26801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:06.334790 26801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:06.334802 26801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:06.336350 26801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45295
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:06.336623 26801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:06.336869 26801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:06.342252 26809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:06.342290 26807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:06.342229 26806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:06.342501 26801 server_base.cc:1034] running on GCE node
I20260620 11:42:06.342828 26801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:06.343138 26801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:06.344334 26801 hybrid_clock.cc:648] HybridClock initialized: now 1781955726344327 us; error 62 us; skew 500 ppm
Jun 20 11:42:06 dist-test-slave-szdr krb5kdc[26788](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955726, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:06.347090 26801 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:42:06.347601 26801 webserver.cc:460] Webserver started at http://127.0.0.1:43201/ using document root <none> and password file <none>
I20260620 11:42:06.347862 26801 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:06.347930 26801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:06.348074 26801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:06.349171 26801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "7cc1d56d25a14cc0b49c51c395f631a1"
format_stamp: "Formatted at 2026-06-20 11:42:06 on dist-test-slave-szdr"
server_key: "c02b508944b76774d40cad438fbe4465"
server_key_iv: "2b01e0028b98ec8e87d70126c5d68249"
server_key_version: "encryptionkey@0"
I20260620 11:42:06.349553 26801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "7cc1d56d25a14cc0b49c51c395f631a1"
format_stamp: "Formatted at 2026-06-20 11:42:06 on dist-test-slave-szdr"
server_key: "c02b508944b76774d40cad438fbe4465"
server_key_iv: "2b01e0028b98ec8e87d70126c5d68249"
server_key_version: "encryptionkey@0"
I20260620 11:42:06.351032 26801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:42:06.351861 26816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:06.352066 26801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:42:06.352161 26801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "7cc1d56d25a14cc0b49c51c395f631a1"
format_stamp: "Formatted at 2026-06-20 11:42:06 on dist-test-slave-szdr"
server_key: "c02b508944b76774d40cad438fbe4465"
server_key_iv: "2b01e0028b98ec8e87d70126c5d68249"
server_key_version: "encryptionkey@0"
I20260620 11:42:06.352244 26801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:06.369793 26801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:06.372045 26801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:06.372233 26801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:06.376556 26801 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45295
I20260620 11:42:06.376600 26868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45295 every 8 connection(s)
I20260620 11:42:06.377005 26801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:06.378209 26869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:06.380616 26869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1: Bootstrap starting.
I20260620 11:42:06.381207 26869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:06.381456 26869 log.cc:824] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:06.382347 26869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1: No bootstrap required, opened a new log
I20260620 11:42:06.384014 26869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260620 11:42:06.384171 26869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:06.384194 26869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc1d56d25a14cc0b49c51c395f631a1, State: Initialized, Role: FOLLOWER
I20260620 11:42:06.384307 26869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260620 11:42:06.384403 26869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:06.384438 26869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:06.384492 26869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:06.384725 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26801
I20260620 11:42:06.384868 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:06.385073 5610 external_mini_cluster.cc:1428] Setting key ea017aa36e9d4d5efe268769a5946e4f
I20260620 11:42:06.386430 26869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260620 11:42:06.386576 26869 leader_election.cc:302] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cc1d56d25a14cc0b49c51c395f631a1; no voters:
I20260620 11:42:06.386821 26869 leader_election.cc:288] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:06.386870 26874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:06.387048 26874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc1d56d25a14cc0b49c51c395f631a1, State: Running, Role: LEADER
I20260620 11:42:06.387102 26869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:06.387185 26874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } }
I20260620 11:42:06.387542 26874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cc1d56d25a14cc0b49c51c395f631a1. Latest consensus state: current_term: 1 leader_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } } }
I20260620 11:42:06.387620 26874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:06.387815 26875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cc1d56d25a14cc0b49c51c395f631a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45295 } } }
I20260620 11:42:06.387969 26875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:06.388373 26880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:06.389022 26880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:06.391109 26880 catalog_manager.cc:1347] Generated new cluster ID: 873643f2d9bc47beaa19d566b37c83a2
I20260620 11:42:06.391178 26880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:42:06 dist-test-slave-szdr krb5kdc[26788](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 11:42:06.397775 26873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:06.385648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:54527 (local address 127.0.0.1:45295)
0620 11:42:06.385861 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:06.385873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:06.385904 (+ 31us) server_negotiation.cc:407] Connection header received
0620 11:42:06.386073 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:06.386079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:06.386129 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:06.386250 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:06.387240 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:06.388809 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:06.389712 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:06.390331 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:06.390457 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:06.393539 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:06.393545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:06.393545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:06.393560 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:06.395777 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:06.396336 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:06.396339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:06.396339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:06.396478 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:06.396747 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:06.396748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:06.396748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:06.397090 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:06.397244 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:06.397436 (+ 192us) server_negotiation.cc:299] Negotiation successful
0620 11:42:06.397544 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":116,"threads_started":1}
I20260620 11:42:06.398916 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:06.403879 26880 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:06.404989 26880 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:06.407850 26873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:06.400491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:54571 (local address 127.0.0.1:45295)
0620 11:42:06.400530 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:06.400532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:06.400669 (+ 137us) server_negotiation.cc:407] Connection header received
0620 11:42:06.400722 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:06.400723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:06.400733 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:06.400790 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:06.401483 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:06.402293 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:06.403356 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:06.403606 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:06.403702 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:06.405044 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:06.405045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:06.405046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:06.405053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:06.406628 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:06.407020 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:06.407022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:06.407022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:06.407080 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:06.407327 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:06.407328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:06.407328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:06.407441 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:06.407570 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:06.407681 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 11:42:06.407745 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 11:42:06.419034 26880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cc1d56d25a14cc0b49c51c395f631a1: Generated new TSK 0
I20260620 11:42:06.419284 26880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:06.425597 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26801
W20260620 11:42:06.430124 26786 connection.cc:535] client connection to 127.0.0.1:45295 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45295): Connection reset by peer (error 104)
2026-06-20T11:42:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:06 dist-test-slave-szdr krb5kdc[26900](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:06 dist-test-slave-szdr krb5kdc[26900](info): set up 2 sockets
Jun 20 11:42:06 dist-test-slave-szdr krb5kdc[26900](info): commencing operation
krb5kdc: starting...
W20260620 11:42:07.486438 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:07 dist-test-slave-szdr krb5kdc[26900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:42:07.613517 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:07.696985 26913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:07.697289 26913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:07.697331 26913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:07.698659 26913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:07.698763 26913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:07.698787 26913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:07.698809 26913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:07.698839 26913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:07.700510 26913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:07.700852 26913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:07.701104 26913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:07.706418 26919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:07.706607 26913 server_base.cc:1034] running on GCE node
W20260620 11:42:07.706422 26918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:07.706418 26921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:07.707031 26913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:07.707320 26913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:07.708489 26913 hybrid_clock.cc:648] HybridClock initialized: now 1781955727708477 us; error 31 us; skew 500 ppm
Jun 20 11:42:07 dist-test-slave-szdr krb5kdc[26900](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:07.711359 26913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:42:07.711874 26913 webserver.cc:460] Webserver started at http://127.0.0.1:43993/ using document root <none> and password file <none>
I20260620 11:42:07.712065 26913 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:07.712109 26913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:07.712201 26913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:07.713416 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "6348dc2ab1414d6eb5490bdcdaef5e53"
format_stamp: "Formatted at 2026-06-20 11:42:07 on dist-test-slave-szdr"
server_key: "eebd2d35a2a1824ecbdfbf6aabc188a8"
server_key_iv: "ecf36c49e5011cdfc29aa113680bcd56"
server_key_version: "encryptionkey@0"
I20260620 11:42:07.713807 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "6348dc2ab1414d6eb5490bdcdaef5e53"
format_stamp: "Formatted at 2026-06-20 11:42:07 on dist-test-slave-szdr"
server_key: "eebd2d35a2a1824ecbdfbf6aabc188a8"
server_key_iv: "ecf36c49e5011cdfc29aa113680bcd56"
server_key_version: "encryptionkey@0"
I20260620 11:42:07.715309 26913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:07.716100 26928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:07.716321 26913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:07.716441 26913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "6348dc2ab1414d6eb5490bdcdaef5e53"
format_stamp: "Formatted at 2026-06-20 11:42:07 on dist-test-slave-szdr"
server_key: "eebd2d35a2a1824ecbdfbf6aabc188a8"
server_key_iv: "ecf36c49e5011cdfc29aa113680bcd56"
server_key_version: "encryptionkey@0"
I20260620 11:42:07.716529 26913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:07.723963 26913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:07.726529 26913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:07.726716 26913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:07.731046 26913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42883
I20260620 11:42:07.731081 26980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42883 every 8 connection(s)
I20260620 11:42:07.731487 26913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:07.732435 26981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:07.734887 26981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53: Bootstrap starting.
I20260620 11:42:07.735476 26981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:07.735709 26981 log.cc:824] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:07.736515 26981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53: No bootstrap required, opened a new log
I20260620 11:42:07.738121 26981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260620 11:42:07.738289 26981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:07.738315 26981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6348dc2ab1414d6eb5490bdcdaef5e53, State: Initialized, Role: FOLLOWER
I20260620 11:42:07.738425 26981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260620 11:42:07.738479 26981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:07.738500 26981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:07.738528 26981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:07.738802 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 26913
I20260620 11:42:07.738917 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:07.739109 5610 external_mini_cluster.cc:1428] Setting key c497071f888ba864e1f5954081eba282
I20260620 11:42:07.739389 26981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260620 11:42:07.739557 26981 leader_election.cc:302] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6348dc2ab1414d6eb5490bdcdaef5e53; no voters:
I20260620 11:42:07.739830 26981 leader_election.cc:288] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:07.739881 26985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:07.740039 26985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [term 1 LEADER]: Becoming Leader. State: Replica: 6348dc2ab1414d6eb5490bdcdaef5e53, State: Running, Role: LEADER
I20260620 11:42:07.740188 26981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:07.740248 26985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } }
I20260620 11:42:07.740651 26985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6348dc2ab1414d6eb5490bdcdaef5e53. Latest consensus state: current_term: 1 leader_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } } }
I20260620 11:42:07.740742 26985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:07.741029 26987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6348dc2ab1414d6eb5490bdcdaef5e53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42883 } } }
I20260620 11:42:07.741137 26987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53 [sys.catalog]: This master's current role is: LEADER
W20260620 11:42:07.742105 27001 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:42:07.742182 27001 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:42:07.742276 26993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:07.742475 26993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:07.744195 26993 catalog_manager.cc:1347] Generated new cluster ID: 9716a7a9a84c4858a062e6245630a1ca
I20260620 11:42:07.744261 26993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:07.745951 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:07.739719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:55487 (local address 127.0.0.1:42883)
0620 11:42:07.739961 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:07.739974 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:07.740016 (+ 42us) server_negotiation.cc:407] Connection header received
0620 11:42:07.740174 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:07.740182 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:07.740250 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:07.740668 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:07.740798 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:07.742392 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:07.743527 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:07.744187 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:07.744690 (+ 503us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:07.745204 (+ 514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:07.745208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:07.745209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:07.745422 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:07.745462 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:07.745477 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:07.745559 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:07.745699 (+ 140us) server_negotiation.cc:299] Negotiation successful
0620 11:42:07.745728 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":94,"threads_started":1}
I20260620 11:42:07.747032 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:07.752441 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:07.748603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:43159 (local address 127.0.0.1:42883)
0620 11:42:07.748638 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:07.748640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:07.748784 (+ 144us) server_negotiation.cc:407] Connection header received
0620 11:42:07.748821 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:07.748823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:07.748838 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:07.748910 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:07.749052 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:07.749879 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:07.750951 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:07.751276 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:07.751356 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:07.752152 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:07.752154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:07.752154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:07.752160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:07.752178 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:07.752182 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:07.752247 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:07.752332 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 11:42:07.752351 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 11:42:07.770488 26993 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:07.771299 26993 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:07.775571 26993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6348dc2ab1414d6eb5490bdcdaef5e53: Generated new TSK 0
I20260620 11:42:07.775777 26993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:07.778308 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 26913
W20260620 11:42:07.782234 26898 connection.cc:535] client connection to 127.0.0.1:42883 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42883): Connection reset by peer (error 104)
2026-06-20T11:42:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:07 dist-test-slave-szdr krb5kdc[27012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:07 dist-test-slave-szdr krb5kdc[27012](info): set up 2 sockets
Jun 20 11:42:07 dist-test-slave-szdr krb5kdc[27012](info): commencing operation
krb5kdc: starting...
W20260620 11:42:08.822273 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:08 dist-test-slave-szdr krb5kdc[27012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 11:42:08.953472 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:09.041013 27025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:09.041293 27025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:09.041335 27025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:09.042734 27025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:09.042824 27025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:09.042848 27025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:09.042869 27025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:09.042901 27025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:09.044623 27025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:09.044953 27025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:09.045218 27025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:09.050921 27030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:09.050916 27031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:09.051112 27025 server_base.cc:1034] running on GCE node
W20260620 11:42:09.050921 27033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:09.051406 27025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:09.051756 27025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:09.052942 27025 hybrid_clock.cc:648] HybridClock initialized: now 1781955729052927 us; error 29 us; skew 500 ppm
Jun 20 11:42:09 dist-test-slave-szdr krb5kdc[27012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:09.055821 27025 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 11:42:09.056360 27025 webserver.cc:460] Webserver started at http://127.0.0.1:46627/ using document root <none> and password file <none>
I20260620 11:42:09.056628 27025 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:09.056710 27025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:09.056869 27025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:09.058127 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "db984233dd85444b825a633bdaae467c"
format_stamp: "Formatted at 2026-06-20 11:42:09 on dist-test-slave-szdr"
server_key: "a64e96c62812891757d722c0453d5105"
server_key_iv: "5bda63507baba500d7ef7d3b35f04d5c"
server_key_version: "encryptionkey@0"
I20260620 11:42:09.058528 27025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "db984233dd85444b825a633bdaae467c"
format_stamp: "Formatted at 2026-06-20 11:42:09 on dist-test-slave-szdr"
server_key: "a64e96c62812891757d722c0453d5105"
server_key_iv: "5bda63507baba500d7ef7d3b35f04d5c"
server_key_version: "encryptionkey@0"
I20260620 11:42:09.059999 27025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:42:09.060806 27040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:09.060999 27025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:09.061110 27025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "db984233dd85444b825a633bdaae467c"
format_stamp: "Formatted at 2026-06-20 11:42:09 on dist-test-slave-szdr"
server_key: "a64e96c62812891757d722c0453d5105"
server_key_iv: "5bda63507baba500d7ef7d3b35f04d5c"
server_key_version: "encryptionkey@0"
I20260620 11:42:09.061187 27025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:09.064759 27025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:09.066902 27025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:09.067072 27025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:09.071241 27025 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33475
I20260620 11:42:09.071271 27092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33475 every 8 connection(s)
I20260620 11:42:09.071622 27025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:09.072667 27093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:09.075217 27093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c: Bootstrap starting.
I20260620 11:42:09.075850 27093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:09.076154 27093 log.cc:824] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:09.076859 27093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c: No bootstrap required, opened a new log
I20260620 11:42:09.078703 27093 raft_consensus.cc:348] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db984233dd85444b825a633bdaae467c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33475 } }
I20260620 11:42:09.078874 27093 raft_consensus.cc:374] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:09.078899 27093 raft_consensus.cc:729] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db984233dd85444b825a633bdaae467c, State: Initialized, Role: FOLLOWER
I20260620 11:42:09.078998 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27025
I20260620 11:42:09.079094 27093 consensus_queue.cc:260] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db984233dd85444b825a633bdaae467c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33475 } }
I20260620 11:42:09.079149 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:09.079210 27093 raft_consensus.cc:388] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:09.079265 27093 raft_consensus.cc:482] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:09.079340 27093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:09.079409 5610 external_mini_cluster.cc:1428] Setting key 8c64bcec0238a33d7dfd08ea6f177b2f
I20260620 11:42:09.080204 27093 raft_consensus.cc:504] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db984233dd85444b825a633bdaae467c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33475 } }
I20260620 11:42:09.080336 27093 leader_election.cc:302] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db984233dd85444b825a633bdaae467c; no voters:
I20260620 11:42:09.080519 27093 leader_election.cc:288] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:09.080559 27097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:09.080749 27097 raft_consensus.cc:686] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [term 1 LEADER]: Becoming Leader. State: Replica: db984233dd85444b825a633bdaae467c, State: Running, Role: LEADER
I20260620 11:42:09.080879 27093 sys_catalog.cc:564] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:09.080884 27097 consensus_queue.cc:237] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db984233dd85444b825a633bdaae467c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33475 } }
I20260620 11:42:09.081367 27100 sys_catalog.cc:455] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [sys.catalog]: SysCatalogTable state changed. Reason: New leader db984233dd85444b825a633bdaae467c. Latest consensus state: current_term: 1 leader_uuid: "db984233dd85444b825a633bdaae467c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db984233dd85444b825a633bdaae467c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33475 } } }
I20260620 11:42:09.081367 27099 sys_catalog.cc:455] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db984233dd85444b825a633bdaae467c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db984233dd85444b825a633bdaae467c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33475 } } }
I20260620 11:42:09.081631 27099 sys_catalog.cc:458] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:09.082041 27100 sys_catalog.cc:458] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:09.082223 27107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:09.082708 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:09.080415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:59823 (local address 127.0.0.1:33475)
0620 11:42:09.080633 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:09.080645 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:09.080713 (+ 68us) server_negotiation.cc:407] Connection header received
0620 11:42:09.080846 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:09.080853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:09.080886 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:09.081076 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:09.081928 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:09.081932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:09.081932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:09.081937 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:09.082016 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:09.082034 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:09.082113 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:09.082383 (+ 270us) server_negotiation.cc:299] Negotiation successful
0620 11:42:09.082435 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260620 11:42:09.082839 27107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:09.083961 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:09.084625 27107 catalog_manager.cc:1347] Generated new cluster ID: 582f4336663d4f87ac29f5f44a5634ca
I20260620 11:42:09.084695 27107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:09.091416 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:09.086345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.229:55325 (local address 127.0.0.1:33475)
0620 11:42:09.086386 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:09.086388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:09.090260 (+ 3872us) server_negotiation.cc:407] Connection header received
0620 11:42:09.090353 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:09.090353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:09.090361 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:09.090449 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:09.091166 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:09.091168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:09.091168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:09.091172 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:09.091189 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:09.091194 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:09.091246 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:09.091289 (+ 43us) server_negotiation.cc:299] Negotiation successful
0620 11:42:09.091307 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:42:09.096843 27107 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:09.097049 27107 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:09.102741 27107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db984233dd85444b825a633bdaae467c: Generated new TSK 0
I20260620 11:42:09.102970 27107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:09.105127 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27025
2026-06-20T11:42:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:09 dist-test-slave-szdr krb5kdc[27124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:09 dist-test-slave-szdr krb5kdc[27124](info): set up 2 sockets
Jun 20 11:42:09 dist-test-slave-szdr krb5kdc[27124](info): commencing operation
krb5kdc: starting...
W20260620 11:42:10.154214 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:10 dist-test-slave-szdr krb5kdc[27124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:10.284687 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37357
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:37357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:10.369812 27137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:10.370113 27137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:10.370154 27137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:10.371369 27137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:10.371457 27137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:10.371482 27137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:10.371503 27137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:10.371531 27137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:10.373126 27137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:37357
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:10.373449 27137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:10.373710 27137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:10.378945 27143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:10.379128 27137 server_base.cc:1034] running on GCE node
W20260620 11:42:10.378981 27142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:10.378947 27145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:10.379572 27137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:10.379928 27137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:10.381119 27137 hybrid_clock.cc:648] HybridClock initialized: now 1781955730381108 us; error 49 us; skew 500 ppm
Jun 20 11:42:10 dist-test-slave-szdr krb5kdc[27124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955730, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:10.387087 27137 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:10.387665 27137 webserver.cc:460] Webserver started at http://127.5.122.190:42523/ using document root <none> and password file <none>
I20260620 11:42:10.387907 27137 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:10.387977 27137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:10.388140 27137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:10.389314 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "4eb75d8efe584dbcb98cf03e81a504a7"
format_stamp: "Formatted at 2026-06-20 11:42:10 on dist-test-slave-szdr"
server_key: "ed1eeecabf6fbb331ca64f927550235c"
server_key_iv: "779b4766c8b982e6b77e9484527fcdde"
server_key_version: "encryptionkey@0"
I20260620 11:42:10.389722 27137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "4eb75d8efe584dbcb98cf03e81a504a7"
format_stamp: "Formatted at 2026-06-20 11:42:10 on dist-test-slave-szdr"
server_key: "ed1eeecabf6fbb331ca64f927550235c"
server_key_iv: "779b4766c8b982e6b77e9484527fcdde"
server_key_version: "encryptionkey@0"
I20260620 11:42:10.391289 27137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:42:10.392221 27152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:10.392465 27137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:10.392583 27137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "4eb75d8efe584dbcb98cf03e81a504a7"
format_stamp: "Formatted at 2026-06-20 11:42:10 on dist-test-slave-szdr"
server_key: "ed1eeecabf6fbb331ca64f927550235c"
server_key_iv: "779b4766c8b982e6b77e9484527fcdde"
server_key_version: "encryptionkey@0"
I20260620 11:42:10.392668 27137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:10.405042 27137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:10.407495 27137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:10.407689 27137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:10.412220 27137 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:37357
I20260620 11:42:10.412257 27204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:37357 every 8 connection(s)
I20260620 11:42:10.412655 27137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:10.413648 27205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:10.416209 27205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7: Bootstrap starting.
I20260620 11:42:10.416860 27205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:10.417162 27205 log.cc:824] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:10.418052 27205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7: No bootstrap required, opened a new log
I20260620 11:42:10.419731 27205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37357 } }
I20260620 11:42:10.419930 27205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:10.419956 27205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eb75d8efe584dbcb98cf03e81a504a7, State: Initialized, Role: FOLLOWER
I20260620 11:42:10.420146 27205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37357 } }
I20260620 11:42:10.420238 27205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:10.420261 27205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:10.420336 27205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:10.420460 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27137
I20260620 11:42:10.420595 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:10.420840 5610 external_mini_cluster.cc:1428] Setting key c734c4e095459119368c65b85f7a0976
I20260620 11:42:10.421739 27205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37357 } }
I20260620 11:42:10.421849 27205 leader_election.cc:302] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eb75d8efe584dbcb98cf03e81a504a7; no voters:
I20260620 11:42:10.422108 27205 leader_election.cc:288] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:10.422142 27210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:10.422389 27210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [term 1 LEADER]: Becoming Leader. State: Replica: 4eb75d8efe584dbcb98cf03e81a504a7, State: Running, Role: LEADER
I20260620 11:42:10.422492 27205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:10.422534 27210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37357 } }
I20260620 11:42:10.423009 27212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eb75d8efe584dbcb98cf03e81a504a7. Latest consensus state: current_term: 1 leader_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37357 } } }
I20260620 11:42:10.423239 27212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:10.423169 27211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb75d8efe584dbcb98cf03e81a504a7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37357 } } }
I20260620 11:42:10.423686 27211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:10.423807 27215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:10.424400 27215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:10.426282 27215 catalog_manager.cc:1347] Generated new cluster ID: 527dda10a403425793a41e4faedf217b
I20260620 11:42:10.426348 27215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:42:10 dist-test-slave-szdr krb5kdc[27124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:10.434638 27209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:10.421471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37255 (local address 127.5.122.190:37357)
0620 11:42:10.421700 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:10.421713 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:10.421756 (+ 43us) server_negotiation.cc:407] Connection header received
0620 11:42:10.421911 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:10.421918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:10.422025 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:10.422198 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:10.423060 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:10.424716 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:10.425777 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:10.426417 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:10.426550 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:10.429855 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:10.429861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:10.429862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:10.429877 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:10.432556 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:10.433116 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:10.433120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:10.433120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:10.433270 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:10.433559 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:10.433560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:10.433560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:10.433900 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:10.434092 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:10.434241 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 11:42:10.434379 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260620 11:42:10.435822 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:10.445541 27209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:10.437479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33231 (local address 127.5.122.190:37357)
0620 11:42:10.437548 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:10.437550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:10.437756 (+ 206us) server_negotiation.cc:407] Connection header received
0620 11:42:10.437807 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:10.437808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:10.437819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:10.437876 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:10.438573 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:10.439304 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:10.440417 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:10.440768 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:10.440833 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:10.442516 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:10.442518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:10.442518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:10.442528 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:10.444184 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:10.444666 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:10.444668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:10.444668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:10.444726 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:10.444967 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:10.444968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:10.444968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:10.445096 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:10.445200 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:10.445365 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 11:42:10.445440 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260620 11:42:10.453994 27215 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:10.454609 27215 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:10.460024 27215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eb75d8efe584dbcb98cf03e81a504a7: Generated new TSK 0
I20260620 11:42:10.460254 27215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:10.468875 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27137
W20260620 11:42:10.473465 27122 connection.cc:535] client connection to 127.5.122.190:37357 recv error: Network error: failed to read from TLS socket (remote: 127.5.122.190:37357): Connection reset by peer (error 104)
2026-06-20T11:42:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:10 dist-test-slave-szdr krb5kdc[27236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:10 dist-test-slave-szdr krb5kdc[27236](info): set up 2 sockets
Jun 20 11:42:10 dist-test-slave-szdr krb5kdc[27236](info): commencing operation
krb5kdc: starting...
W20260620 11:42:11.522222 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:11 dist-test-slave-szdr krb5kdc[27236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:11.649313 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37333
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:37333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:11.734460 27249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:11.734699 27249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:11.734725 27249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:11.735950 27249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:11.735999 27249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:11.736013 27249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:11.736027 27249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:11.736039 27249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:11.737551 27249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:37333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:11.737773 27249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:11.737998 27249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:11.743145 27255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:11.743183 27254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:11.743183 27257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:11.743216 27249 server_base.cc:1034] running on GCE node
I20260620 11:42:11.743709 27249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:11.744050 27249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:11.745246 27249 hybrid_clock.cc:648] HybridClock initialized: now 1781955731745213 us; error 64 us; skew 500 ppm
Jun 20 11:42:11 dist-test-slave-szdr krb5kdc[27236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955731, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:11.748102 27249 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:11.748540 27249 webserver.cc:460] Webserver started at http://127.5.122.190:44761/ using document root <none> and password file <none>
I20260620 11:42:11.748741 27249 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:11.748785 27249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:11.748888 27249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:11.750054 27249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "136a6f4d961f45b58bc06886c3499cc7"
format_stamp: "Formatted at 2026-06-20 11:42:11 on dist-test-slave-szdr"
server_key: "af53d18b9b5e99a05516814f8049c37a"
server_key_iv: "96e538478c73b325a2698541dbfa1f84"
server_key_version: "encryptionkey@0"
I20260620 11:42:11.750386 27249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "136a6f4d961f45b58bc06886c3499cc7"
format_stamp: "Formatted at 2026-06-20 11:42:11 on dist-test-slave-szdr"
server_key: "af53d18b9b5e99a05516814f8049c37a"
server_key_iv: "96e538478c73b325a2698541dbfa1f84"
server_key_version: "encryptionkey@0"
I20260620 11:42:11.751755 27249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:42:11.752687 27264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:11.752936 27249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 11:42:11.753044 27249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "136a6f4d961f45b58bc06886c3499cc7"
format_stamp: "Formatted at 2026-06-20 11:42:11 on dist-test-slave-szdr"
server_key: "af53d18b9b5e99a05516814f8049c37a"
server_key_iv: "96e538478c73b325a2698541dbfa1f84"
server_key_version: "encryptionkey@0"
I20260620 11:42:11.753113 27249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:11.769135 27249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:11.772053 27249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:11.772188 27249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:11.776517 27249 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:37333
I20260620 11:42:11.776556 27316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:37333 every 8 connection(s)
I20260620 11:42:11.776959 27249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:11.777972 27317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:11.780462 27317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7: Bootstrap starting.
I20260620 11:42:11.781078 27317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:11.781381 27317 log.cc:824] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:11.782228 27317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7: No bootstrap required, opened a new log
I20260620 11:42:11.784005 27317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136a6f4d961f45b58bc06886c3499cc7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37333 } }
I20260620 11:42:11.784196 27317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:11.784245 27317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 136a6f4d961f45b58bc06886c3499cc7, State: Initialized, Role: FOLLOWER
I20260620 11:42:11.784411 27317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136a6f4d961f45b58bc06886c3499cc7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37333 } }
I20260620 11:42:11.784498 27317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:11.784543 27317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:11.784592 27317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:11.784911 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27249
I20260620 11:42:11.785040 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:11.785223 5610 external_mini_cluster.cc:1428] Setting key 8579fba1b174b38a7f3cab65aa63e950
I20260620 11:42:11.785913 27317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136a6f4d961f45b58bc06886c3499cc7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37333 } }
I20260620 11:42:11.786062 27317 leader_election.cc:302] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 136a6f4d961f45b58bc06886c3499cc7; no voters:
I20260620 11:42:11.786243 27317 leader_election.cc:288] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:11.786269 27322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:11.786391 27322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [term 1 LEADER]: Becoming Leader. State: Replica: 136a6f4d961f45b58bc06886c3499cc7, State: Running, Role: LEADER
I20260620 11:42:11.786545 27317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:11.786528 27322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136a6f4d961f45b58bc06886c3499cc7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37333 } }
I20260620 11:42:11.787029 27324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 136a6f4d961f45b58bc06886c3499cc7. Latest consensus state: current_term: 1 leader_uuid: "136a6f4d961f45b58bc06886c3499cc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136a6f4d961f45b58bc06886c3499cc7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37333 } } }
I20260620 11:42:11.787135 27324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:11.787355 27323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "136a6f4d961f45b58bc06886c3499cc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "136a6f4d961f45b58bc06886c3499cc7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 37333 } } }
I20260620 11:42:11.787678 27323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:11.787837 27330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:11.788507 27330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:11.790374 27330 catalog_manager.cc:1347] Generated new cluster ID: 5e8c97c5f8d8441d9d0904ae483be88b
I20260620 11:42:11.790459 27330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:11.791756 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:11.785804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51623 (local address 127.5.122.190:37333)
0620 11:42:11.786132 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:11.786162 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:11.786197 (+ 35us) server_negotiation.cc:407] Connection header received
0620 11:42:11.786362 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:11.786370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:11.786447 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:11.786817 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:11.786945 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:11.788419 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:11.789305 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:11.789924 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:11.790073 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:11.791051 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:11.791055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:11.791056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:11.791254 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:11.791286 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:11.791297 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:11.791399 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:11.791485 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 11:42:11.791515 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":134,"threads_started":1}
I20260620 11:42:11.792737 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:11.801259 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:11.794161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53321 (local address 127.5.122.190:37333)
0620 11:42:11.794202 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:11.794205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:11.798208 (+ 4003us) server_negotiation.cc:407] Connection header received
0620 11:42:11.798298 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:11.798298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:11.798324 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:11.798381 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:11.798558 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:11.799107 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:11.799872 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:11.800105 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:11.800186 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:11.800963 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:11.800965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:11.800965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:11.800971 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:11.800987 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:11.800991 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:11.801047 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:11.801126 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 11:42:11.801139 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:42:11.815840 27330 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:11.816452 27330 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:11.820664 27330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 136a6f4d961f45b58bc06886c3499cc7: Generated new TSK 0
I20260620 11:42:11.820920 27330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:11.825452 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27249
W20260620 11:42:11.829931 27234 connection.cc:535] client connection to 127.5.122.190:37333 recv error: Network error: failed to read from TLS socket (remote: 127.5.122.190:37333): Connection reset by peer (error 104)
2026-06-20T11:42:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:11 dist-test-slave-szdr krb5kdc[27348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:11 dist-test-slave-szdr krb5kdc[27348](info): set up 2 sockets
Jun 20 11:42:11 dist-test-slave-szdr krb5kdc[27348](info): commencing operation
krb5kdc: starting...
W20260620 11:42:12.862272 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:12 dist-test-slave-szdr krb5kdc[27348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:12.998306 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36375
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:13.087023 27361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:13.087344 27361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:13.087392 27361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:13.088855 27361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:13.088958 27361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:13.088984 27361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:13.089008 27361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:13.089042 27361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:13.091063 27361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:36375
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:13.091445 27361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:13.091737 27361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:13.097404 27367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:13.097395 27369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:13.097395 27366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:13.097731 27361 server_base.cc:1034] running on GCE node
I20260620 11:42:13.098069 27361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:13.098433 27361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:13.099608 27361 hybrid_clock.cc:648] HybridClock initialized: now 1781955733099579 us; error 51 us; skew 500 ppm
Jun 20 11:42:13 dist-test-slave-szdr krb5kdc[27348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955733, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:13.102486 27361 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:13.102928 27361 webserver.cc:460] Webserver started at http://127.5.122.190:39635/ using document root <none> and password file <none>
I20260620 11:42:13.103142 27361 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:13.103189 27361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:13.103370 27361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:13.104468 27361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "2f812510200c42748a5cf6ea4bc1dff4"
format_stamp: "Formatted at 2026-06-20 11:42:13 on dist-test-slave-szdr"
server_key: "4b7b3d5e6a64ecc99eec326c0b3bd05c"
server_key_iv: "e0d64e579f5f4f216d29105654fa6847"
server_key_version: "encryptionkey@0"
I20260620 11:42:13.104776 27361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "2f812510200c42748a5cf6ea4bc1dff4"
format_stamp: "Formatted at 2026-06-20 11:42:13 on dist-test-slave-szdr"
server_key: "4b7b3d5e6a64ecc99eec326c0b3bd05c"
server_key_iv: "e0d64e579f5f4f216d29105654fa6847"
server_key_version: "encryptionkey@0"
I20260620 11:42:13.106225 27361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260620 11:42:13.107034 27376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:13.107255 27361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:13.107370 27361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "2f812510200c42748a5cf6ea4bc1dff4"
format_stamp: "Formatted at 2026-06-20 11:42:13 on dist-test-slave-szdr"
server_key: "4b7b3d5e6a64ecc99eec326c0b3bd05c"
server_key_iv: "e0d64e579f5f4f216d29105654fa6847"
server_key_version: "encryptionkey@0"
I20260620 11:42:13.107472 27361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:13.111035 27361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:13.113483 27361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:13.113649 27361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:13.117663 27361 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:36375
I20260620 11:42:13.117724 27428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:36375 every 8 connection(s)
I20260620 11:42:13.118134 27361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:13.119203 27429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:13.121666 27429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4: Bootstrap starting.
I20260620 11:42:13.122279 27429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:13.122514 27429 log.cc:824] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:13.123207 27429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4: No bootstrap required, opened a new log
I20260620 11:42:13.123703 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27361
I20260620 11:42:13.123894 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:13.124074 5610 external_mini_cluster.cc:1428] Setting key 61511774404ec6e3b4c618462111fa76
I20260620 11:42:13.125787 27429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f812510200c42748a5cf6ea4bc1dff4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36375 } }
I20260620 11:42:13.126058 27429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:13.126121 27429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f812510200c42748a5cf6ea4bc1dff4, State: Initialized, Role: FOLLOWER
I20260620 11:42:13.126260 27429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f812510200c42748a5cf6ea4bc1dff4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36375 } }
I20260620 11:42:13.126387 27429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:13.126439 27429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:13.126490 27429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:13.127148 27433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:13.124622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50503 (local address 127.5.122.190:36375)
0620 11:42:13.124794 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:13.124803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:13.124828 (+ 25us) server_negotiation.cc:407] Connection header received
0620 11:42:13.125693 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:13.125701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:13.125730 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:13.125855 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:13.126695 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:13.126699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:13.126699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:13.126703 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:13.126741 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:13.126751 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:13.126808 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:13.126898 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 11:42:13.126951 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260620 11:42:13.127231 27429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f812510200c42748a5cf6ea4bc1dff4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36375 } }
I20260620 11:42:13.127355 27429 leader_election.cc:302] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f812510200c42748a5cf6ea4bc1dff4; no voters:
I20260620 11:42:13.127518 27429 leader_election.cc:288] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:13.127574 27434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:13.127772 27429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:13.127885 27434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [term 1 LEADER]: Becoming Leader. State: Replica: 2f812510200c42748a5cf6ea4bc1dff4, State: Running, Role: LEADER
I20260620 11:42:13.128028 27434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f812510200c42748a5cf6ea4bc1dff4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36375 } }
I20260620 11:42:13.128217 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:13.129875 27435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f812510200c42748a5cf6ea4bc1dff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f812510200c42748a5cf6ea4bc1dff4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36375 } } }
I20260620 11:42:13.129987 27435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:13.130125 27435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f812510200c42748a5cf6ea4bc1dff4. Latest consensus state: current_term: 1 leader_uuid: "2f812510200c42748a5cf6ea4bc1dff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f812510200c42748a5cf6ea4bc1dff4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 36375 } } }
I20260620 11:42:13.130182 27435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:13.130242 27453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:13.130781 27453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:13.131068 27433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:13.129679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50827 (local address 127.5.122.190:36375)
0620 11:42:13.129718 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:13.129719 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:13.129938 (+ 219us) server_negotiation.cc:407] Connection header received
0620 11:42:13.130035 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:13.130036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:13.130040 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:13.130089 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:13.130827 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:13.130828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:13.130829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:13.130833 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:13.130848 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:13.130853 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:13.130900 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:13.130976 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 11:42:13.131011 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:42:13.132654 27453 catalog_manager.cc:1347] Generated new cluster ID: 59e0a179260647b1920942703306354d
I20260620 11:42:13.132717 27453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:13.143261 27453 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:13.143466 27453 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:13.145256 27453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f812510200c42748a5cf6ea4bc1dff4: Generated new TSK 0
I20260620 11:42:13.145448 27453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:13.150014 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27361
2026-06-20T11:42:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:13 dist-test-slave-szdr krb5kdc[27461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:13 dist-test-slave-szdr krb5kdc[27461](info): set up 2 sockets
Jun 20 11:42:13 dist-test-slave-szdr krb5kdc[27461](info): commencing operation
krb5kdc: starting...
W20260620 11:42:14.194440 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:14 dist-test-slave-szdr krb5kdc[27461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:14.325670 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:34571
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:39759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:34571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:14.412853 27474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:14.413105 27474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:14.413133 27474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:14.414357 27474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:14.414402 27474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:14.414417 27474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:14.414429 27474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:14.414441 27474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:14.415961 27474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:39759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:34571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:34571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:14.416186 27474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:14.416388 27474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:14.421646 27480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:14.421819 27474 server_base.cc:1034] running on GCE node
W20260620 11:42:14.421659 27479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:14.421655 27482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:14.422209 27474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:14.422497 27474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:14.423717 27474 hybrid_clock.cc:648] HybridClock initialized: now 1781955734423699 us; error 34 us; skew 500 ppm
Jun 20 11:42:14 dist-test-slave-szdr krb5kdc[27461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955734, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:14.426749 27474 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:14.427286 27474 webserver.cc:460] Webserver started at http://127.5.122.190:39399/ using document root <none> and password file <none>
I20260620 11:42:14.427543 27474 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:14.427597 27474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:14.427733 27474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:14.428857 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "8980cc6e4d7546dcafef914d92267261"
format_stamp: "Formatted at 2026-06-20 11:42:14 on dist-test-slave-szdr"
server_key: "f0dee1e72c72bdb9892a826e973df064"
server_key_iv: "5230194457838e6b1ca144b25fb9f87d"
server_key_version: "encryptionkey@0"
I20260620 11:42:14.429231 27474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "8980cc6e4d7546dcafef914d92267261"
format_stamp: "Formatted at 2026-06-20 11:42:14 on dist-test-slave-szdr"
server_key: "f0dee1e72c72bdb9892a826e973df064"
server_key_iv: "5230194457838e6b1ca144b25fb9f87d"
server_key_version: "encryptionkey@0"
I20260620 11:42:14.430822 27474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:14.431710 27489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:14.431915 27474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:14.432021 27474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "8980cc6e4d7546dcafef914d92267261"
format_stamp: "Formatted at 2026-06-20 11:42:14 on dist-test-slave-szdr"
server_key: "f0dee1e72c72bdb9892a826e973df064"
server_key_iv: "5230194457838e6b1ca144b25fb9f87d"
server_key_version: "encryptionkey@0"
I20260620 11:42:14.432108 27474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:14.454066 27474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:14.456672 27474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:14.456828 27474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:14.461071 27474 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:34571
I20260620 11:42:14.461130 27541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:34571 every 8 connection(s)
I20260620 11:42:14.461596 27474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:14.462584 27542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:14.465003 27542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261: Bootstrap starting.
I20260620 11:42:14.465638 27542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:14.465930 27542 log.cc:824] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:14.466688 27542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261: No bootstrap required, opened a new log
I20260620 11:42:14.468325 27542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8980cc6e4d7546dcafef914d92267261" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34571 } }
I20260620 11:42:14.468513 27542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:14.468583 27542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8980cc6e4d7546dcafef914d92267261, State: Initialized, Role: FOLLOWER
I20260620 11:42:14.468757 27542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8980cc6e4d7546dcafef914d92267261" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34571 } }
I20260620 11:42:14.468840 27542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:14.468884 27542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:14.468935 27542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:14.469813 27542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8980cc6e4d7546dcafef914d92267261" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34571 } }
I20260620 11:42:14.469936 27542 leader_election.cc:302] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8980cc6e4d7546dcafef914d92267261; no voters:
I20260620 11:42:14.470155 27542 leader_election.cc:288] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:14.470256 27545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:14.470427 27542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:14.470448 27545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [term 1 LEADER]: Becoming Leader. State: Replica: 8980cc6e4d7546dcafef914d92267261, State: Running, Role: LEADER
I20260620 11:42:14.470687 27545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8980cc6e4d7546dcafef914d92267261" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34571 } }
I20260620 11:42:14.471091 27547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8980cc6e4d7546dcafef914d92267261. Latest consensus state: current_term: 1 leader_uuid: "8980cc6e4d7546dcafef914d92267261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8980cc6e4d7546dcafef914d92267261" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34571 } } }
I20260620 11:42:14.471112 27546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8980cc6e4d7546dcafef914d92267261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8980cc6e4d7546dcafef914d92267261" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34571 } } }
I20260620 11:42:14.471174 27547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:14.471208 27546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:14.471437 27554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:14.471529 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27474
I20260620 11:42:14.471669 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:14.471894 5610 external_mini_cluster.cc:1428] Setting key daf4cbcd06589793a300a844bd17da4e
I20260620 11:42:14.472046 27554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:14.473786 27554 catalog_manager.cc:1347] Generated new cluster ID: 4c774d7527a54d12b10ec3710417ceea
I20260620 11:42:14.473877 27554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 20 11:42:14 dist-test-slave-szdr krb5kdc[27461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:14.484413 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:14.472495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44419 (local address 127.5.122.190:34571)
0620 11:42:14.472707 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:14.472717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:14.472752 (+ 35us) server_negotiation.cc:407] Connection header received
0620 11:42:14.472873 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:14.472880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:14.472939 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:14.473081 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:14.473856 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:14.475282 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:14.476384 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:14.477094 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:14.477228 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:14.480382 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:14.480388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:14.480388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:14.480404 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:14.482575 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:14.483032 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:14.483035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:14.483035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:14.483159 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:14.483385 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:14.483386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:14.483387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:14.483728 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:14.483867 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:14.484006 (+ 139us) server_negotiation.cc:299] Negotiation successful
0620 11:42:14.484162 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260620 11:42:14.485644 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:14.494731 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:14.487164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40719 (local address 127.5.122.190:34571)
0620 11:42:14.487198 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:14.487200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:14.487310 (+ 110us) server_negotiation.cc:407] Connection header received
0620 11:42:14.487370 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:14.487371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:14.487381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:14.487438 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:14.488097 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:14.488859 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:14.489901 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:14.490232 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:14.490292 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:14.491808 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:14.491809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:14.491810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:14.491817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:14.493489 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:14.493899 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:14.493900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:14.493901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:14.493986 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:14.494177 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:14.494178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:14.494178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:14.494306 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:14.494394 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:14.494543 (+ 149us) server_negotiation.cc:299] Negotiation successful
0620 11:42:14.494632 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 11:42:14.510094 27554 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:14.510886 27554 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:14.528775 27554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8980cc6e4d7546dcafef914d92267261: Generated new TSK 0
I20260620 11:42:14.529028 27554 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:14.538079 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27474
2026-06-20T11:42:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:14 dist-test-slave-szdr krb5kdc[27573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:14 dist-test-slave-szdr krb5kdc[27573](info): set up 2 sockets
Jun 20 11:42:14 dist-test-slave-szdr krb5kdc[27573](info): commencing operation
krb5kdc: starting...
W20260620 11:42:15.586529 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:15 dist-test-slave-szdr krb5kdc[27573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:15.715328 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:45661
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:45661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:15.808614 27586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:15.808902 27586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:15.808948 27586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:15.810325 27586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:15.810410 27586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:15.810436 27586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:15.810464 27586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:15.810495 27586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:15.812078 27586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:45661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:45661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:15.812390 27586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:15.812644 27586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:15.818027 27591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:15.818022 27594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:15.818022 27592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:15.818094 27586 server_base.cc:1034] running on GCE node
I20260620 11:42:15.818560 27586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:15.818892 27586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:15.820078 27586 hybrid_clock.cc:648] HybridClock initialized: now 1781955735820066 us; error 46 us; skew 500 ppm
Jun 20 11:42:15 dist-test-slave-szdr krb5kdc[27573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955735, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:15.822901 27586 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:15.823431 27586 webserver.cc:460] Webserver started at http://127.5.122.190:41829/ using document root <none> and password file <none>
I20260620 11:42:15.823684 27586 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:15.823758 27586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:15.823910 27586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:15.825047 27586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "ea824dac875f4f438f16b3a5de05a7be"
format_stamp: "Formatted at 2026-06-20 11:42:15 on dist-test-slave-szdr"
server_key: "0d79299db83dd1cc660b8d301a6378f2"
server_key_iv: "c53c2310a8fccfa797eaed330a4312ce"
server_key_version: "encryptionkey@0"
I20260620 11:42:15.825438 27586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "ea824dac875f4f438f16b3a5de05a7be"
format_stamp: "Formatted at 2026-06-20 11:42:15 on dist-test-slave-szdr"
server_key: "0d79299db83dd1cc660b8d301a6378f2"
server_key_iv: "c53c2310a8fccfa797eaed330a4312ce"
server_key_version: "encryptionkey@0"
I20260620 11:42:15.827047 27586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:15.827982 27601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:15.828197 27586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:15.828311 27586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "ea824dac875f4f438f16b3a5de05a7be"
format_stamp: "Formatted at 2026-06-20 11:42:15 on dist-test-slave-szdr"
server_key: "0d79299db83dd1cc660b8d301a6378f2"
server_key_iv: "c53c2310a8fccfa797eaed330a4312ce"
server_key_version: "encryptionkey@0"
I20260620 11:42:15.828393 27586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:15.838195 27586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:15.840601 27586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:15.840814 27586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:15.845300 27586 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:45661
I20260620 11:42:15.845333 27653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:45661 every 8 connection(s)
I20260620 11:42:15.845757 27586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:15.846809 27654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:15.849211 27654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be: Bootstrap starting.
I20260620 11:42:15.849824 27654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:15.850194 27654 log.cc:824] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:15.851123 27654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be: No bootstrap required, opened a new log
I20260620 11:42:15.851140 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27586
I20260620 11:42:15.851238 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:15.851524 5610 external_mini_cluster.cc:1428] Setting key 275303b79217fbe64c21a71a304952d8
I20260620 11:42:15.853288 27654 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea824dac875f4f438f16b3a5de05a7be" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45661 } }
I20260620 11:42:15.853483 27654 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:15.853576 27654 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea824dac875f4f438f16b3a5de05a7be, State: Initialized, Role: FOLLOWER
I20260620 11:42:15.853746 27654 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea824dac875f4f438f16b3a5de05a7be" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45661 } }
I20260620 11:42:15.853848 27654 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:15.853883 27654 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:15.853986 27654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:15.854874 27654 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea824dac875f4f438f16b3a5de05a7be" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45661 } }
I20260620 11:42:15.855003 27654 leader_election.cc:302] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea824dac875f4f438f16b3a5de05a7be; no voters:
I20260620 11:42:15.855183 27654 leader_election.cc:288] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:15.855247 27659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:15.855412 27659 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [term 1 LEADER]: Becoming Leader. State: Replica: ea824dac875f4f438f16b3a5de05a7be, State: Running, Role: LEADER
I20260620 11:42:15.855479 27654 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:15.855571 27659 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea824dac875f4f438f16b3a5de05a7be" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45661 } }
I20260620 11:42:15.856086 27661 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea824dac875f4f438f16b3a5de05a7be. Latest consensus state: current_term: 1 leader_uuid: "ea824dac875f4f438f16b3a5de05a7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea824dac875f4f438f16b3a5de05a7be" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45661 } } }
I20260620 11:42:15.856091 27660 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea824dac875f4f438f16b3a5de05a7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea824dac875f4f438f16b3a5de05a7be" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 45661 } } }
I20260620 11:42:15.856208 27661 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:15.856213 27660 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:15.856433 27668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:15.857009 27668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:15.858824 27668 catalog_manager.cc:1347] Generated new cluster ID: 044fb5c485954cfab504ab48cda8507c
I20260620 11:42:15.858882 27668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:15.859303 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:15.852100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50711 (local address 127.5.122.190:45661)
0620 11:42:15.852307 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:15.852315 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:15.852343 (+ 28us) server_negotiation.cc:407] Connection header received
0620 11:42:15.853278 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:15.853288 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:15.853353 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:15.853555 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:15.853816 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:15.855635 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:15.856930 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:15.857614 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:15.857736 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:15.858567 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:15.858572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:15.858572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:15.858785 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:15.858826 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:15.858842 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:15.858912 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:15.859027 (+ 115us) server_negotiation.cc:299] Negotiation successful
0620 11:42:15.859100 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260620 11:42:15.860502 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:15.865713 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:15.862154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46163 (local address 127.5.122.190:45661)
0620 11:42:15.862196 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:15.862198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:15.862255 (+ 57us) server_negotiation.cc:407] Connection header received
0620 11:42:15.862316 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:15.862317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:15.862328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:15.862386 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:15.862540 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:15.863284 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:15.864305 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:15.864616 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:15.864694 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:15.865442 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:15.865443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:15.865444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:15.865449 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:15.865468 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:15.865472 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:15.865532 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:15.865592 (+ 60us) server_negotiation.cc:299] Negotiation successful
0620 11:42:15.865620 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:42:15.892768 27668 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:15.893368 27668 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:15.904945 27668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea824dac875f4f438f16b3a5de05a7be: Generated new TSK 0
I20260620 11:42:15.905194 27668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:15.910607 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27586
2026-06-20T11:42:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:15 dist-test-slave-szdr krb5kdc[27685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:15 dist-test-slave-szdr krb5kdc[27685](info): set up 2 sockets
Jun 20 11:42:15 dist-test-slave-szdr krb5kdc[27685](info): commencing operation
krb5kdc: starting...
W20260620 11:42:16.994351 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:17 dist-test-slave-szdr krb5kdc[27685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:17.121551 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41427
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:43903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:41427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:17.207139 27698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:17.207438 27698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:17.207485 27698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:17.208806 27698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:17.208897 27698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:17.208921 27698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:17.208943 27698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:17.208974 27698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:17.210700 27698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:43903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:41427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:41427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:17.211032 27698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:17.211297 27698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:17.216910 27704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:17.216886 27706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:17.217098 27698 server_base.cc:1034] running on GCE node
W20260620 11:42:17.216890 27703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:17.217442 27698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:17.217744 27698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:17.218923 27698 hybrid_clock.cc:648] HybridClock initialized: now 1781955737218911 us; error 61 us; skew 500 ppm
Jun 20 11:42:17 dist-test-slave-szdr krb5kdc[27685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955737, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:17.221735 27698 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:17.222327 27698 webserver.cc:460] Webserver started at http://127.5.122.190:44979/ using document root <none> and password file <none>
I20260620 11:42:17.222587 27698 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:17.222657 27698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:17.222813 27698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:17.223948 27698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "fde9a7196d664e6cab0081cc82a502e4"
format_stamp: "Formatted at 2026-06-20 11:42:17 on dist-test-slave-szdr"
server_key: "f6bbc6655df26288745f3ff12453359a"
server_key_iv: "732de710c064db784379d1bac5b4d7ec"
server_key_version: "encryptionkey@0"
I20260620 11:42:17.224350 27698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "fde9a7196d664e6cab0081cc82a502e4"
format_stamp: "Formatted at 2026-06-20 11:42:17 on dist-test-slave-szdr"
server_key: "f6bbc6655df26288745f3ff12453359a"
server_key_iv: "732de710c064db784379d1bac5b4d7ec"
server_key_version: "encryptionkey@0"
I20260620 11:42:17.225788 27698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:17.226773 27713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:17.227015 27698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:17.227129 27698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "fde9a7196d664e6cab0081cc82a502e4"
format_stamp: "Formatted at 2026-06-20 11:42:17 on dist-test-slave-szdr"
server_key: "f6bbc6655df26288745f3ff12453359a"
server_key_iv: "732de710c064db784379d1bac5b4d7ec"
server_key_version: "encryptionkey@0"
I20260620 11:42:17.227218 27698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:17.231207 27698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:17.233502 27698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:17.233685 27698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:17.238576 27698 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:41427
I20260620 11:42:17.238616 27765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:41427 every 8 connection(s)
I20260620 11:42:17.239012 27698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:17.240302 27766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:17.242880 27766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4: Bootstrap starting.
I20260620 11:42:17.243520 27766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:17.243842 27766 log.cc:824] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:17.244623 27766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4: No bootstrap required, opened a new log
I20260620 11:42:17.246447 27766 raft_consensus.cc:348] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde9a7196d664e6cab0081cc82a502e4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41427 } }
I20260620 11:42:17.246603 27766 raft_consensus.cc:374] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 11:42:17.246627 27766 raft_consensus.cc:729] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fde9a7196d664e6cab0081cc82a502e4, State: Initialized, Role: FOLLOWER
I20260620 11:42:17.246807 27766 consensus_queue.cc:260] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde9a7196d664e6cab0081cc82a502e4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41427 } }
I20260620 11:42:17.246882 27766 raft_consensus.cc:388] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 11:42:17.246905 27766 raft_consensus.cc:482] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 11:42:17.246959 27766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:17.247236 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27698
I20260620 11:42:17.247350 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:17.247579 5610 external_mini_cluster.cc:1428] Setting key dc91ec4f77d848a25e7515db0e791fb0
I20260620 11:42:17.248322 27766 raft_consensus.cc:504] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde9a7196d664e6cab0081cc82a502e4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41427 } }
I20260620 11:42:17.248453 27766 leader_election.cc:302] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fde9a7196d664e6cab0081cc82a502e4; no voters:
I20260620 11:42:17.248687 27766 leader_election.cc:288] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 11:42:17.248709 27771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:17.248826 27771 raft_consensus.cc:686] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [term 1 LEADER]: Becoming Leader. State: Replica: fde9a7196d664e6cab0081cc82a502e4, State: Running, Role: LEADER
I20260620 11:42:17.248993 27766 sys_catalog.cc:564] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:17.248970 27771 consensus_queue.cc:237] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde9a7196d664e6cab0081cc82a502e4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41427 } }
I20260620 11:42:17.249910 27773 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fde9a7196d664e6cab0081cc82a502e4. Latest consensus state: current_term: 1 leader_uuid: "fde9a7196d664e6cab0081cc82a502e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde9a7196d664e6cab0081cc82a502e4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41427 } } }
I20260620 11:42:17.250036 27773 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:17.250217 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:17.248131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44193 (local address 127.5.122.190:41427)
0620 11:42:17.248355 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:17.248364 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:17.248468 (+ 104us) server_negotiation.cc:407] Connection header received
0620 11:42:17.248627 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:17.248635 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:17.248669 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:17.249140 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:17.249633 (+ 493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:17.249637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:17.249637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:17.249642 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:17.249694 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:17.249713 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:17.249779 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:17.249850 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 11:42:17.249904 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":127,"threads_started":1}
I20260620 11:42:17.250444 27782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:17.250458 27772 sys_catalog.cc:455] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fde9a7196d664e6cab0081cc82a502e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fde9a7196d664e6cab0081cc82a502e4" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 41427 } } }
I20260620 11:42:17.250945 27772 sys_catalog.cc:458] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:17.251274 27782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:17.251478 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:17.253803 27782 catalog_manager.cc:1347] Generated new cluster ID: 3743402bd7cf47018047d9e23b09e0b2
I20260620 11:42:17.253862 27782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:17.254529 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:17.253099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46909 (local address 127.5.122.190:41427)
0620 11:42:17.253138 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:17.253140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:17.253457 (+ 317us) server_negotiation.cc:407] Connection header received
0620 11:42:17.253535 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:17.253536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:17.253542 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:17.253592 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:17.254285 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:17.254286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:17.254286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 11:42:17.254291 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:17.254307 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 11:42:17.254312 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:17.254371 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:17.254428 (+ 57us) server_negotiation.cc:299] Negotiation successful
0620 11:42:17.254444 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:42:17.273682 27782 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:17.273939 27782 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:17.277537 27782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fde9a7196d664e6cab0081cc82a502e4: Generated new TSK 0
I20260620 11:42:17.277778 27782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:17.283361 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27698
2026-06-20T11:42:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1372 ms)
[----------] 18 tests from AuthTokenIssuingTest (24351 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:17 dist-test-slave-szdr krb5kdc[27797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:17 dist-test-slave-szdr krb5kdc[27797](info): set up 2 sockets
Jun 20 11:42:17 dist-test-slave-szdr krb5kdc[27797](info): commencing operation
krb5kdc: starting...
W20260620 11:42:18.334328 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:18.461223 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:34439
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:34439,127.5.122.189:36017,127.5.122.188:37533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:18.545840 27810 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 11:42:18.546160 27810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:18.546191 27810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:18.546257 27810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:18.547523 27810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:18.547622 27810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:18.547647 27810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:18.547669 27810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:18.547698 27810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:18.549396 27810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:34439,127.5.122.189:36017,127.5.122.188:37533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:34439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:18.549728 27810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:18.550038 27810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:18.555419 27815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:18.555440 27810 server_base.cc:1034] running on GCE node
W20260620 11:42:18.555439 27816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:18.555421 27818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:18.555945 27810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:18.556278 27810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:18.557461 27810 hybrid_clock.cc:648] HybridClock initialized: now 1781955738557446 us; error 29 us; skew 500 ppm
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:18.560129 27810 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:18.560653 27810 webserver.cc:460] Webserver started at http://127.5.122.190:45585/ using document root <none> and password file <none>
I20260620 11:42:18.560912 27810 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:18.560985 27810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:18.561138 27810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:18.562348 27810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "6063ad756d0e49d2a412dc7fdd895d30"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "f54d668899cc4d916588d64fdcafbbd1"
server_key_iv: "267f6c7b34738890677f0ffdad7c5808"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.562758 27810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "6063ad756d0e49d2a412dc7fdd895d30"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "f54d668899cc4d916588d64fdcafbbd1"
server_key_iv: "267f6c7b34738890677f0ffdad7c5808"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.564234 27810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:42:18.565120 27825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:18.565311 27810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:18.565451 27810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "6063ad756d0e49d2a412dc7fdd895d30"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "f54d668899cc4d916588d64fdcafbbd1"
server_key_iv: "267f6c7b34738890677f0ffdad7c5808"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.565536 27810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:18.599502 27810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:18.601604 27810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:18.601783 27810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:18.605891 27810 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:34439
I20260620 11:42:18.605926 27877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:34439 every 8 connection(s)
I20260620 11:42:18.606364 27810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:18.606886 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27810
I20260620 11:42:18.607018 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:18.607251 5610 external_mini_cluster.cc:1428] Setting key df674ca2b3e667bb4fa2fc65f68591fb
I20260620 11:42:18.607740 27878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:18.609736 27878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.122.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.189@KRBTEST.COM" created.
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:18.623046 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.610608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.5.122.190:34439)
0620 11:42:18.610836 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.610841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.610900 (+ 59us) server_negotiation.cc:407] Connection header received
0620 11:42:18.611504 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.611507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.611527 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.611636 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.612233 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.613100 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.614077 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.614429 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.614935 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.619119 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.619137 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.619137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.619145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.621086 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.621612 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.621622 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.621622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.621718 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.622027 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.622028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.622028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.622362 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.622469 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.622750 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.622863 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
I20260620 11:42:18.623169 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.610567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:34439 (local address 127.0.0.1:35726)
0620 11:42:18.610762 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.610809 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.611335 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.611640 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.611643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.611941 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.612112 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.612116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.613205 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.613206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.613912 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.613913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.614809 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.614820 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.614930 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.615622 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.615632 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.618978 (+ 3346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.621203 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.621212 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.621214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.621461 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.621815 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.621816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.621816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.621918 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.622468 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.622476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.622577 (+ 101us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.622691 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.622977 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":88,"thread_start_us":115,"threads_started":1}
I20260620 11:42:18.624293 27878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:18.624766 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.624582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50556)
0620 11:42:18.624641 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.624727 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 11:42:18.624933 27827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
I20260620 11:42:18.625429 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.625320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50572)
0620 11:42:18.625372 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.625404 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 11:42:18.625633 27878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:36017: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab.
Entry for principal kudu/127.5.122.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab.
I20260620 11:42:18.637785 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:36017
--webserver_interface=127.5.122.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab
--principal=kudu/127.5.122.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:34439,127.5.122.189:36017,127.5.122.188:37533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:42:18.674930 27878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } attempt: 1
I20260620 11:42:18.675588 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.675391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50574)
0620 11:42:18.675483 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.675542 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 11:42:18.675992 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.675894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50582)
0620 11:42:18.675938 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.675971 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 11:42:18.676107 27878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:36017: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
W20260620 11:42:18.722736 27885 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 11:42:18.722996 27885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:18.723016 27885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:18.723040 27885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:18.724349 27885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:18.724416 27885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:18.724431 27885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:18.724443 27885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:18.724457 27885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:18.726193 27885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:34439,127.5.122.189:36017,127.5.122.188:37533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab
--principal=kudu/127.5.122.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:36017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:18.726507 27885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:18.726778 27885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:18.732326 27893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:18.732301 27890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:18.732633 27885 server_base.cc:1034] running on GCE node
W20260620 11:42:18.732348 27891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:18.732906 27885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:18.733251 27885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:18.734503 27885 hybrid_clock.cc:648] HybridClock initialized: now 1781955738734478 us; error 37 us; skew 500 ppm
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:18.737298 27885 init.cc:377] Logged in from keytab as kudu/127.5.122.189@KRBTEST.COM (short username kudu)
I20260620 11:42:18.737757 27885 webserver.cc:460] Webserver started at http://127.5.122.189:35033/ using document root <none> and password file <none>
I20260620 11:42:18.738025 27885 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:18.738090 27885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:18.738198 27885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:18.739336 27885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data/instance:
uuid: "5ad78eb4787341c39131cb292bf17f24"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "8cd5887665a3f4a8302e48dc0c64b7a7"
server_key_iv: "f005b2d42ecb9f0459f92f4bad4ea089"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.739686 27885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/wal/instance:
uuid: "5ad78eb4787341c39131cb292bf17f24"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "8cd5887665a3f4a8302e48dc0c64b7a7"
server_key_iv: "f005b2d42ecb9f0459f92f4bad4ea089"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.741503 27885 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260620 11:42:18.742465 27900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:18.742693 27885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260620 11:42:18.742797 27885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/wal
uuid: "5ad78eb4787341c39131cb292bf17f24"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "8cd5887665a3f4a8302e48dc0c64b7a7"
server_key_iv: "f005b2d42ecb9f0459f92f4bad4ea089"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.742880 27885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:18.744249 27878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } attempt: 2
I20260620 11:42:18.744697 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.744563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50592)
0620 11:42:18.744625 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.744667 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 11:42:18.745121 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.745034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50608)
0620 11:42:18.745080 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.745105 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260620 11:42:18.745232 27878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:36017: Network error: Client connection negotiation failed: client connection to 127.5.122.189:36017: connect: Connection refused (error 111)
I20260620 11:42:18.753437 27885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:18.756595 27885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:18.756850 27885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:18.760999 27885 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.189:36017
I20260620 11:42:18.761034 27952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.189:36017 every 8 connection(s)
I20260620 11:42:18.761466 27885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
I20260620 11:42:18.762625 27953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:18.763124 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27885
I20260620 11:42:18.763247 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-1/wal/instance
I20260620 11:42:18.763440 5610 external_mini_cluster.cc:1428] Setting key a6ffa25c4f89de821a0462f6264e9d8d
I20260620 11:42:18.764484 27953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.122.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.188@KRBTEST.COM" created.
I20260620 11:42:18.776979 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.765789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35738 (local address 127.5.122.190:34439)
0620 11:42:18.765878 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.765881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.765895 (+ 14us) server_negotiation.cc:407] Connection header received
0620 11:42:18.766579 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.766580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.766597 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.766667 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.767363 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.768194 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.769518 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.769911 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.770016 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.772575 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.772576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.772576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.772584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.774126 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.774608 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.774609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.774609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.774678 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.775004 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.775006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.775006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.775166 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.775249 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.776763 (+ 1514us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.776876 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:42:18.777208 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.765437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:34439 (local address 127.0.0.1:35738)
0620 11:42:18.765693 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.765781 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.766464 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.766751 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.766755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.767033 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.767249 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.767254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.768318 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.768319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.769414 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.769418 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.770071 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.770073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.770203 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.770808 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.770817 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.772464 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.774233 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.774242 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.774245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.774525 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.774763 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.774763 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.774763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.774913 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.775241 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.775249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.776580 (+ 1331us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.776736 (+ 156us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.776944 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260620 11:42:18.777781 27953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:42:18.785647 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.778128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50622)
0620 11:42:18.778196 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.778207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.778253 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.778543 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.778544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.778736 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.778803 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.778805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.779676 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.779678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.780195 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.780196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.780686 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.780687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.780756 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.781244 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.781249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.782729 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.784502 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.784503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.784504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.784676 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.784918 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.784919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.784919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.784973 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.785393 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.785393 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.785436 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.785491 (+ 55us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.785573 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 11:42:18.785809 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.778210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50622 (local address 127.5.122.189:36017)
0620 11:42:18.778400 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.778404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.778423 (+ 19us) server_negotiation.cc:407] Connection header received
0620 11:42:18.778441 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.778443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.778455 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.778543 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.778873 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.779584 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.780281 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.780607 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.780691 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.782845 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.782853 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.782854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.782861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.784407 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.784778 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.784782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.784782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.784846 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.785024 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.785024 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.785025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.785296 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.785394 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.785563 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.785651 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260620 11:42:18.786486 27953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:18.786903 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.786784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48064)
0620 11:42:18.786833 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.786883 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 11:42:18.787045 27902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
I20260620 11:42:18.787508 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.787405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48076)
0620 11:42:18.787459 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.787484 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260620 11:42:18.787618 27953 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:37533: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab.
Entry for principal kudu/127.5.122.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab.
I20260620 11:42:18.793820 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:37533
--webserver_interface=127.5.122.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab
--principal=kudu/127.5.122.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:36353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:34439,127.5.122.189:36017,127.5.122.188:37533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:42:18.836426 27878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } attempt: 3
I20260620 11:42:18.836876 27953 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } attempt: 1
I20260620 11:42:18.837318 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.837168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48088)
0620 11:42:18.837232 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.837273 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 11:42:18.838074 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.837934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48100)
0620 11:42:18.838009 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.838044 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260620 11:42:18.838176 27953 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:37533: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:42:18.844925 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.836974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.5.122.189:36017)
0620 11:42:18.837021 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.837024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.837132 (+ 108us) server_negotiation.cc:407] Connection header received
0620 11:42:18.837172 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.837173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.837184 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.837267 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.837728 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.838469 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.839528 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.839756 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.839819 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.842086 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.842088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.842088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.842096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.843670 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.844060 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.844061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.844062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.844124 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.844401 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.844402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.844402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.844533 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.844592 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.844763 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.844835 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:42:18.844952 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.836931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50636)
0620 11:42:18.837006 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.837020 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.837088 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.837303 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.837305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.837510 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.837599 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.837602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.838560 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.838562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.839426 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.839430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.839837 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.839838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.839926 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.840471 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.840477 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.841914 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.843774 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.843775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.843776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.843978 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.844235 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.844237 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.844237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.844320 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.844603 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.844604 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.844661 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.844734 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.844853 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260620 11:42:18.845857 27878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:18.846269 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.846151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48112)
0620 11:42:18.846206 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.846242 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 11:42:18.846593 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.846505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48126)
0620 11:42:18.846550 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.846575 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 11:42:18.846755 27878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:37533: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
I20260620 11:42:18.877938 27878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } attempt: 1
I20260620 11:42:18.878576 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.878424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48134)
0620 11:42:18.878488 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.878537 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 11:42:18.878979 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.878875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48148)
0620 11:42:18.878922 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.878954 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 11:42:18.879120 27878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:37533: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
W20260620 11:42:18.882541 27960 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 11:42:18.882776 27960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:18.882794 27960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:18.882817 27960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:18.884073 27960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:18.884119 27960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:18.884135 27960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:18.884148 27960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:18.884161 27960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:18.885758 27960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:36353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:34439,127.5.122.189:36017,127.5.122.188:37533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab
--principal=kudu/127.5.122.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:37533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:18.886073 27960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:18.886332 27960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:18.891714 27968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:18.891907 27960 server_base.cc:1034] running on GCE node
W20260620 11:42:18.891714 27965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:18.891739 27966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:18.892355 27960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:18.892691 27960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:18.893862 27960 hybrid_clock.cc:648] HybridClock initialized: now 1781955738893839 us; error 41 us; skew 500 ppm
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:18.896960 27960 init.cc:377] Logged in from keytab as kudu/127.5.122.188@KRBTEST.COM (short username kudu)
I20260620 11:42:18.897564 27960 webserver.cc:460] Webserver started at http://127.5.122.188:38317/ using document root <none> and password file <none>
I20260620 11:42:18.897832 27960 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:18.897908 27960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:18.898094 27960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:18.899309 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data/instance:
uuid: "90d0c148cde54e128006cd86364332c7"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "ae55cfd53b0fc81aa9d0daa91e54d9d2"
server_key_iv: "23276df17d2c5f4268672b469717a7d7"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.899721 27960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/wal/instance:
uuid: "90d0c148cde54e128006cd86364332c7"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "ae55cfd53b0fc81aa9d0daa91e54d9d2"
server_key_iv: "23276df17d2c5f4268672b469717a7d7"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.901198 27960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260620 11:42:18.902094 27975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:18.902290 27960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:18.902401 27960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/wal
uuid: "90d0c148cde54e128006cd86364332c7"
format_stamp: "Formatted at 2026-06-20 11:42:18 on dist-test-slave-szdr"
server_key: "ae55cfd53b0fc81aa9d0daa91e54d9d2"
server_key_iv: "23276df17d2c5f4268672b469717a7d7"
server_key_version: "encryptionkey@0"
I20260620 11:42:18.902489 27960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:18.906306 27953 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } attempt: 2
I20260620 11:42:18.906797 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.906655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48156)
0620 11:42:18.906720 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.906766 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 11:42:18.907127 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.907044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48162)
0620 11:42:18.907086 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.907111 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260620 11:42:18.907222 27953 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:37533: Network error: Client connection negotiation failed: client connection to 127.5.122.188:37533: connect: Connection refused (error 111)
I20260620 11:42:18.914381 27960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:18.916769 27960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:18.916903 27960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:18.921456 27960 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.188:37533
I20260620 11:42:18.921514 28027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.188:37533 every 8 connection(s)
I20260620 11:42:18.921949 27960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
I20260620 11:42:18.923168 28028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:18.924887 28028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:18.929282 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 27960
I20260620 11:42:18.929435 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781955466149030-5610-0/minicluster-data/master-2/wal/instance
I20260620 11:42:18.929675 5610 external_mini_cluster.cc:1428] Setting key 847fe5ff1125e23083faf083347ef3f8
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:18.935777 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.925775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.5.122.190:34439)
0620 11:42:18.925842 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.925844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.926117 (+ 273us) server_negotiation.cc:407] Connection header received
0620 11:42:18.926690 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.926691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.926703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.926765 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.927404 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.927880 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.929240 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.929480 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.929546 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.932378 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.932380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.932381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.932392 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.934085 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.934672 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.934673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.934674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.934735 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.935041 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.935042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.935042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.935150 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.935201 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.935573 (+ 372us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.935646 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260620 11:42:18.935987 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.925739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:34439 (local address 127.0.0.1:35748)
0620 11:42:18.925945 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.926023 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.926593 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.926828 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.926831 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.927104 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.927326 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.927331 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.928036 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.928039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.929120 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.929124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.929603 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.929605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.929800 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.930504 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.930515 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.932265 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.934225 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.934232 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.934235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.934582 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.934807 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.934808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.934808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.934967 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.935217 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.935224 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.935426 (+ 202us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.935554 (+ 128us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.935742 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":119,"threads_started":1}
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:42:18.936846 28028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:18.940158 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.930230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.5.122.188:37533)
0620 11:42:18.930479 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.930484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.930617 (+ 133us) server_negotiation.cc:407] Connection header received
0620 11:42:18.930645 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.930647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.930672 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.930775 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.931498 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.932289 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.933274 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.933786 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.933886 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.936702 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.936709 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.936709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.936718 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.938798 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.939264 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.939268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.939268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.939327 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.939550 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.939551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.939551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.939702 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.939819 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.939954 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.940037 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":119,"threads_started":1}
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:42:18.945452 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.937562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.5.122.189:36017)
0620 11:42:18.937631 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.937633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.937642 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:42:18.937676 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.937676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.937684 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.937739 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.938216 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.938977 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.939943 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.940344 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.940420 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.942768 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.942769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.942770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.942777 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.944347 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.944697 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.944699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.944699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.944770 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.944925 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.944926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.944927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.945082 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.945139 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.945309 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.945381 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260620 11:42:18.945499 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.937438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:36017 (local address 127.0.0.1:50646)
0620 11:42:18.937491 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.937502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.937539 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.937765 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.937767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.938002 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.938051 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.938053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.939057 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.939058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.939838 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.939841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.940417 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.940418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.940486 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.940972 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.940978 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.942661 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.944456 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.944457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.944457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.944621 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.944840 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.944841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.944841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.944886 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.945158 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.945158 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.945208 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.945286 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.945396 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:18.946513 28028 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:18.949007 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.941063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.5.122.190:34439)
0620 11:42:18.941103 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.941105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.941113 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:42:18.941178 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.941179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.941189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.941231 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.941822 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.942533 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.943312 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.943520 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.943599 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.946169 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.946171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.946171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.946187 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.947774 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.948138 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.948139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.948140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.948210 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.948445 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.948446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.948447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.948574 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.948779 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.948817 (+ 38us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.948897 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:42:18.954308 27878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } attempt: 2
I20260620 11:42:18.955611 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.946798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48182)
0620 11:42:18.946835 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.946845 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.946880 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.947860 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.947861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.947971 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.948012 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.948014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.948676 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.948677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.949489 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.949492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.950068 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.950069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.950130 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.950690 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.950695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.952059 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.954272 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.954273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.954274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.954496 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.954942 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.954943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.954943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.955010 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.955335 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.955336 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.955383 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.955454 (+ 71us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.955536 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260620 11:42:18.955986 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.946838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.5.122.188:37533)
0620 11:42:18.947064 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.947066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.947072 (+ 6us) server_negotiation.cc:407] Connection header received
0620 11:42:18.947089 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.947090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.947100 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.947177 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.948091 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.948591 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.949593 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.949946 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.950044 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.952169 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.952171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.952173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.952186 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.954155 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.954579 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.954581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.954581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.954857 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.955094 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.955095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.955095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.955239 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.955307 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.955513 (+ 206us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.955599 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260620 11:42:18.958424 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.949872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50652 (local address 127.5.122.189:36017)
0620 11:42:18.949923 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.949925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.949935 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:42:18.949991 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.949992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.950003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.950052 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.950584 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.951054 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.951774 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.952052 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.952134 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.954823 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.954824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.954825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.954833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.956725 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.957199 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.957201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.957201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.957266 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.957507 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.957509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.957509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.957665 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.958246 (+ 581us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.958283 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.958361 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:42:18.958644 28028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: Bootstrap starting.
Jun 20 11:42:18 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:42:18.959627 28028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:18.960042 28028 log.cc:824] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:18.960649 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:18.960983 28028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: No bootstrap required, opened a new log
I20260620 11:42:18.962745 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.954617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48186)
0620 11:42:18.954673 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.954684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.954722 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.955014 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.955015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.955201 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.955261 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.955264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.956556 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.956557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.957121 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.957124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.957625 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.957626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:18.957706 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.958141 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:18.958148 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:18.959537 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:18.961467 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.961469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.961470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.961727 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.961936 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:18.961937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:18.961938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:18.962048 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:18.962403 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:18.962405 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:18.962458 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 11:42:18.962541 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 11:42:18.962652 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260620 11:42:18.962855 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.954650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.5.122.188:37533)
0620 11:42:18.954856 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.954858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.954865 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:42:18.954884 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.954885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.954896 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.954994 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.955711 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.956463 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.957228 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.957456 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.957574 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.959630 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.959632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.959632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.959641 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.961377 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.961800 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.961801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.961801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.961868 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.962122 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.962123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.962123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.962318 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.962403 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.962594 (+ 191us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.962690 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":138,"threads_started":1}
I20260620 11:42:18.962198 27852 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6063ad756d0e49d2a412dc7fdd895d30"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35762
I20260620 11:42:18.963501 28028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:18.963646 28028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90d0c148cde54e128006cd86364332c7, State: Initialized, Role: FOLLOWER
I20260620 11:42:18.963747 28028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:18.964115 28028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:18.964071 28035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:18.964203 28035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:18.964664 27878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30: Bootstrap starting.
I20260620 11:42:18.965355 27878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:18.965720 27878 log.cc:824] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30: Log is configured to *not* fsync() on all Append() calls
W20260620 11:42:18.966223 28046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:42:18.966341 28046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:42:18.966662 27878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30: No bootstrap required, opened a new log
I20260620 11:42:18.967819 27878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:18.967979 27878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6063ad756d0e49d2a412dc7fdd895d30, State: Initialized, Role: FOLLOWER
I20260620 11:42:18.968153 27878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:18.968529 28049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:18.968606 27878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:18.968680 28049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:18.966784 27960 master_runner.cc:186] Error getting master registration for 127.5.122.190:34439: OK, instance_id { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" instance_seqno: 1781955738604686 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 11:42:18.969426 27960 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 11:42:18.969559 27852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 11:42:18.969630 27852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:18.970557 27852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:18.970827 27852 leader_election.cc:288] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [CANDIDATE]: Term 1 election: Requested vote from peers 5ad78eb4787341c39131cb292bf17f24 (127.5.122.189:36017), 90d0c148cde54e128006cd86364332c7 (127.5.122.188:37533)
I20260620 11:42:18.972286 28002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6063ad756d0e49d2a412dc7fdd895d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "90d0c148cde54e128006cd86364332c7"
I20260620 11:42:18.972431 28002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:18.973537 28002 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6063ad756d0e49d2a412dc7fdd895d30 in term 1.
I20260620 11:42:18.973817 27827 leader_election.cc:302] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6063ad756d0e49d2a412dc7fdd895d30, 90d0c148cde54e128006cd86364332c7; no voters:
I20260620 11:42:18.973946 28049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:18.974613 28049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [term 1 LEADER]: Becoming Leader. State: Replica: 6063ad756d0e49d2a412dc7fdd895d30, State: Running, Role: LEADER
I20260620 11:42:18.974742 28049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:18.975704 28049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6063ad756d0e49d2a412dc7fdd895d30. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:18.975791 28049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:18.972314 27810 master_runner.cc:186] Error getting master registration for 127.5.122.189:36017: OK, instance_id { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" instance_seqno: 1781955738759831 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 11:42:18.978510 27810 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 11:42:18.979298 28060 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:42:18.979393 28060 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:42:18.979547 28070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:18.979746 28070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:18.981431 28002 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [term 1 FOLLOWER]: Refusing update from remote peer 6063ad756d0e49d2a412dc7fdd895d30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:42:18.981901 28049 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:42:18.983718 28035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6063ad756d0e49d2a412dc7fdd895d30. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:18.983835 28035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:18.986395 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.973549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.5.122.189:36017)
0620 11:42:18.973600 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.973602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.974561 (+ 959us) server_negotiation.cc:407] Connection header received
0620 11:42:18.974679 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.974680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.974693 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.974728 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.975610 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.976448 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.977605 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.977948 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.978038 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.982031 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.982032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.982033 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.982040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.984065 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.984560 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.984562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.984563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.984647 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.984996 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.984997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.984997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.985157 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.985716 (+ 559us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.986166 (+ 450us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.986282 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:42:18.987110 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.973214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.5.122.190:34439)
0620 11:42:18.973263 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.973265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.973555 (+ 290us) server_negotiation.cc:407] Connection header received
0620 11:42:18.973710 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.973725 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.973738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.973768 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.975151 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.975929 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.977549 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.977860 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.977906 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:18.981886 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:18.981887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:18.981888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:18.981896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:18.985394 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.985870 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.985871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.985872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.985944 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:18.986206 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:18.986207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:18.986207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:18.986355 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:18.986438 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:18.986919 (+ 481us) server_negotiation.cc:299] Negotiation successful
0620 11:42:18.987009 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260620 11:42:18.988955 28049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:18.989081 28049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:18.989693 28035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:18.989789 28035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:18.990193 28073 mvcc.cc:205] Tried to move back new op lower bound from 7298890706865115136 to 7298890706841563136. Current Snapshot: MvccSnapshot[applied={T|T < 7298890706865115136}]
I20260620 11:42:18.990578 28072 mvcc.cc:205] Tried to move back new op lower bound from 7298890706865115136 to 7298890706841563136. Current Snapshot: MvccSnapshot[applied={T|T < 7298890706865115136}]
I20260620 11:42:18.992161 28070 catalog_manager.cc:1347] Generated new cluster ID: 9d7502e2c9704c88aa039df54e0607f5
I20260620 11:42:18.992229 28070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:18.998343 27953 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } attempt: 3
Jun 20 11:42:19 dist-test-slave-szdr krb5kdc[27797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955738, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:42:19.008266 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.998681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:37533 (local address 127.0.0.1:48206)
0620 11:42:18.998791 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:18.998803 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:18.998847 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:18.999156 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:18.999158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:18.999345 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:18.999394 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:18.999396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:19.000384 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:19.000386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:19.001208 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:19.001210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:19.002864 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:19.002865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:19.002935 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:19.003374 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:19.003379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:19.004900 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:19.007107 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:19.007109 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:19.007109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:19.007343 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:19.007563 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:19.007564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:19.007564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:19.007632 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:19.007930 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:19.007931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:19.007979 (+ 48us) client_negotiation.cc:769] Sending connection context
0620 11:42:19.008049 (+ 70us) client_negotiation.cc:240] Negotiation successful
0620 11:42:19.008160 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260620 11:42:19.008679 28032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.998943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.5.122.188:37533)
0620 11:42:18.998984 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.998986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.998994 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:42:18.999020 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.999020 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.999030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.999076 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.999476 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:19.000278 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:19.001567 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:19.001940 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:19.002034 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:19.005003 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:19.005005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:19.005005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:19.005031 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:19.006987 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:19.007435 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:19.007436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:19.007437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:19.007496 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:19.007698 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:19.007698 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:19.007699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:19.007840 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:19.007901 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:19.008482 (+ 581us) server_negotiation.cc:299] Negotiation successful
0620 11:42:19.008574 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:42:19.010262 28070 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:19.010545 27953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: Bootstrap starting.
I20260620 11:42:19.011092 28070 catalog_manager.cc:1504] Loading token signing keys...
I20260620 11:42:19.011416 27953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:19.011744 27953 log.cc:824] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:19.013360 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:18.974343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.5.122.188:37533)
0620 11:42:18.974389 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:18.974392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:18.986853 (+ 12461us) server_negotiation.cc:407] Connection header received
0620 11:42:18.988305 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:18.988307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:18.988321 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:18.988357 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:18.989354 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.990251 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.991499 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:18.991893 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:18.991944 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:19.000379 (+ 8435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:19.000381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:19.000381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:19.000390 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:19.002756 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:19.012473 (+ 9717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:19.012474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:19.012474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:19.012534 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:19.012731 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:19.012731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:19.012732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:19.012870 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:19.012956 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:19.013165 (+ 209us) server_negotiation.cc:299] Negotiation successful
0620 11:42:19.013257 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:42:19.014858 27953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: No bootstrap required, opened a new log
I20260620 11:42:19.017123 27953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:19.018265 27953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad78eb4787341c39131cb292bf17f24, State: Initialized, Role: FOLLOWER
I20260620 11:42:19.018409 27953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } }
I20260620 11:42:19.018811 27953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:19.018914 27927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:19.019970 27927 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [term 1 FOLLOWER]: Refusing update from remote peer 6063ad756d0e49d2a412dc7fdd895d30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:42:19.020186 27926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6063ad756d0e49d2a412dc7fdd895d30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5ad78eb4787341c39131cb292bf17f24"
I20260620 11:42:19.020339 28076 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:42:19.020991 27926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6063ad756d0e49d2a412dc7fdd895d30 in term 1.
I20260620 11:42:19.024638 28079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:19.024761 28079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:19.024832 28079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6063ad756d0e49d2a412dc7fdd895d30. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:19.024883 28079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:19.024916 28079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:19.024971 28079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:19.025041 28070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30: Generated new TSK 0
I20260620 11:42:19.025250 28070 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 11:42:19.026093 28090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:42:19.026170 28090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:42:19.026505 28092 mvcc.cc:205] Tried to move back new op lower bound from 7298890706978869248 to 7298890706841563136. Current Snapshot: MvccSnapshot[applied={T|T < 7298890706865115136}]
I20260620 11:42:19.029018 28049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6063ad756d0e49d2a412dc7fdd895d30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6063ad756d0e49d2a412dc7fdd895d30" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 34439 } } peers { permanent_uuid: "5ad78eb4787341c39131cb292bf17f24" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 36017 } } peers { permanent_uuid: "90d0c148cde54e128006cd86364332c7" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 37533 } } }
I20260620 11:42:19.029120 28049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6063ad756d0e49d2a412dc7fdd895d30 [sys.catalog]: This master's current role is: LEADER
W20260620 11:42:19.059702 27882 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.057840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.5.122.190:34439)
0620 11:42:19.057893 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.057895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.058937 (+ 1042us) server_negotiation.cc:407] Connection header received
0620 11:42:19.059049 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.059051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.059063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.059095 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.059608 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35766: BlockingRecv error: recv got EOF from 127.0.0.1:35766 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260620 11:42:19.060662 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.058358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50658 (local address 127.5.122.189:36017)
0620 11:42:19.058398 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.058400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.060111 (+ 1711us) server_negotiation.cc:407] Connection header received
0620 11:42:19.060214 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.060216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.060227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.060258 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.060578 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50658: BlockingRecv error: recv got EOF from 127.0.0.1:50658 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 11:42:19.061546 28033 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.058668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.5.122.188:37533)
0620 11:42:19.058710 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.058711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.061041 (+ 2330us) server_negotiation.cc:407] Connection header received
0620 11:42:19.061140 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.061142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.061155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.061184 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.061485 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48218: BlockingRecv error: recv got EOF from 127.0.0.1:48218 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 11:42:19.064981 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.063357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50674 (local address 127.5.122.189:36017)
0620 11:42:19.063394 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.063396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.064396 (+ 1000us) server_negotiation.cc:407] Connection header received
0620 11:42:19.064524 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.064525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.064546 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.064577 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.064929 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50674: BlockingRecv error: recv got EOF from 127.0.0.1:50674 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260620 11:42:19.068768 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.067675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50682 (local address 127.5.122.189:36017)
0620 11:42:19.067715 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.067717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.067964 (+ 247us) server_negotiation.cc:407] Connection header received
0620 11:42:19.068068 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.068069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.068081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.068110 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.068458 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50682: BlockingRecv error: recv got EOF from 127.0.0.1:50682 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 11:42:19.072940 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.072070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.5.122.189:36017)
0620 11:42:19.072111 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.072113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.072366 (+ 253us) server_negotiation.cc:407] Connection header received
0620 11:42:19.072466 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.072467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.072478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.072508 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.072885 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50686: BlockingRecv error: recv got EOF from 127.0.0.1:50686 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260620 11:42:19.081213 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.080235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.5.122.189:36017)
0620 11:42:19.080280 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.080282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.080571 (+ 289us) server_negotiation.cc:407] Connection header received
0620 11:42:19.080693 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.080695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.080708 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.080740 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.081154 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50690: BlockingRecv error: recv got EOF from 127.0.0.1:50690 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 11:42:19.092062 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.090823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.5.122.189:36017)
0620 11:42:19.090871 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.090873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.091187 (+ 314us) server_negotiation.cc:407] Connection header received
0620 11:42:19.091471 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.091474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.091490 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.091527 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.091994 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50692: BlockingRecv error: recv got EOF from 127.0.0.1:50692 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 11:42:19.110590 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.109564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50698 (local address 127.5.122.189:36017)
0620 11:42:19.109611 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.109613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.109916 (+ 303us) server_negotiation.cc:407] Connection header received
0620 11:42:19.110060 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.110061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.110074 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.110105 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.110518 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50698: BlockingRecv error: recv got EOF from 127.0.0.1:50698 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 11:42:19.145177 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.144064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50702 (local address 127.5.122.189:36017)
0620 11:42:19.144114 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.144116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.144382 (+ 266us) server_negotiation.cc:407] Connection header received
0620 11:42:19.144492 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.144493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.144506 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.144537 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.145113 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50702: BlockingRecv error: recv got EOF from 127.0.0.1:50702 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 11:42:19.211825 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.210996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50712 (local address 127.5.122.189:36017)
0620 11:42:19.211059 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.211061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.211217 (+ 156us) server_negotiation.cc:407] Connection header received
0620 11:42:19.211296 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.211298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.211312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.211380 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.211752 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50712: BlockingRecv error: recv got EOF from 127.0.0.1:50712 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260620 11:42:19.342007 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.341255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.5.122.189:36017)
0620 11:42:19.341385 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.341388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.341418 (+ 30us) server_negotiation.cc:407] Connection header received
0620 11:42:19.341499 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.341500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.341511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.341577 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.341904 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50724: BlockingRecv error: recv got EOF from 127.0.0.1:50724 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260620 11:42:19.600296 27958 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:19.599454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50730 (local address 127.5.122.189:36017)
0620 11:42:19.599536 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:19.599538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:19.599548 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:42:19.599639 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:19.599640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:19.599650 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:19.599724 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:19.600227 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50730: BlockingRecv error: recv got EOF from 127.0.0.1:50730 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 11:42:19.966854 28046 catalog_manager.cc:1259] Loaded cluster ID: 9d7502e2c9704c88aa039df54e0607f5
I20260620 11:42:19.966941 28046 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: loading cluster ID for follower catalog manager: success
I20260620 11:42:19.967813 28046 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: acquiring CA information for follower catalog manager: success
I20260620 11:42:19.968145 28046 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90d0c148cde54e128006cd86364332c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 11:42:20.026666 28090 catalog_manager.cc:1259] Loaded cluster ID: 9d7502e2c9704c88aa039df54e0607f5
I20260620 11:42:20.026757 28090 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: loading cluster ID for follower catalog manager: success
I20260620 11:42:20.027642 28090 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: acquiring CA information for follower catalog manager: success
I20260620 11:42:20.027951 28090 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ad78eb4787341c39131cb292bf17f24: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 11:42:20.116333 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:20.113612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.5.122.189:36017)
0620 11:42:20.113763 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:20.113765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:20.113774 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:42:20.113799 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:20.113800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:20.113852 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:20.113948 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:20.114140 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.114658 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.115474 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.115684 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.115763 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:20.115961 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:20.116076 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:20.116170 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:20.116196 (+ 26us) server_negotiation.cc:299] Negotiation successful
0620 11:42:20.116219 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
I20260620 11:42:20.120940 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:20.118483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.5.122.188:37533)
0620 11:42:20.118593 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:20.118595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:20.118602 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:42:20.118637 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:20.118638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:20.118674 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:20.118745 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:20.118844 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.119445 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.120167 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.120416 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.120471 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:20.120610 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:20.120715 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:20.120775 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:20.120805 (+ 30us) server_negotiation.cc:299] Negotiation successful
0620 11:42:20.120821 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
I20260620 11:42:20.125921 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:20.123142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50756 (local address 127.5.122.189:36017)
0620 11:42:20.123199 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:20.123201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:20.123210 (+ 9us) server_negotiation.cc:407] Connection header received
0620 11:42:20.123242 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:20.123243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:20.123256 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:20.123312 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:20.123480 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.124257 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.125088 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.125392 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.125464 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:20.125612 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:20.125651 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:20.125703 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:20.125807 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 11:42:20.125841 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260620 11:42:20.125985 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:20.123258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48234 (local address 127.5.122.188:37533)
0620 11:42:20.123300 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:20.123302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:20.123312 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:42:20.123376 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:20.123377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:20.123388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:20.123444 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:20.123590 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.124085 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.125134 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.125414 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.125470 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:20.125680 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:20.125713 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:20.125741 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:20.125866 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 11:42:20.125891 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:42:20.126425 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:20.122826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.5.122.190:34439)
0620 11:42:20.122946 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:20.122948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:20.123044 (+ 96us) server_negotiation.cc:407] Connection header received
0620 11:42:20.123338 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:20.123339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:20.123393 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:20.123475 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:20.123680 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.124431 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.125353 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:20.125701 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:20.125768 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:20.126016 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:20.126137 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:20.126179 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:20.126303 (+ 124us) server_negotiation.cc:299] Negotiation successful
0620 11:42:20.126330 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260620 11:42:20.128104 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27810
I20260620 11:42:20.133510 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27885
I20260620 11:42:20.138757 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 27960
2026-06-20T11:42:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2856 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 11:42:20 dist-test-slave-szdr krb5kdc[28179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 11:42:20 dist-test-slave-szdr krb5kdc[28179](info): set up 2 sockets
Jun 20 11:42:20 dist-test-slave-szdr krb5kdc[28179](info): commencing operation
krb5kdc: starting...
W20260620 11:42:21.186480 5610 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/test-user.keytab.
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T11:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T11:42:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.122.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
Entry for principal kudu/127.5.122.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab.
I20260620 11:42:21.321954 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:40575
--webserver_interface=127.5.122.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:40575,127.5.122.189:39717,127.5.122.188:43785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 11:42:21.405651 28192 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 11:42:21.405927 28192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:21.405961 28192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:21.406025 28192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:21.407231 28192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:21.407310 28192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:21.407335 28192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:21.407367 28192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:21.407388 28192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:21.408901 28192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:40575,127.5.122.189:39717,127.5.122.188:43785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.190.keytab
--principal=kudu/127.5.122.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.190:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.122.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:21.409199 28192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:21.409440 28192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:21.414750 28198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:21.414817 28192 server_base.cc:1034] running on GCE node
W20260620 11:42:21.414819 28200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:21.414750 28197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:21.415215 28192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:21.415534 28192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:21.416725 28192 hybrid_clock.cc:648] HybridClock initialized: now 1781955741416710 us; error 45 us; skew 500 ppm
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:21.422909 28192 init.cc:377] Logged in from keytab as kudu/127.5.122.190@KRBTEST.COM (short username kudu)
I20260620 11:42:21.423429 28192 webserver.cc:460] Webserver started at http://127.5.122.190:33925/ using document root <none> and password file <none>
I20260620 11:42:21.423620 28192 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:21.423663 28192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:21.423763 28192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:21.424803 28192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data/instance:
uuid: "a2594ee63db746f59e98c890cc1e03d7"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "272369108bdd150428cc2c1ef5a85398"
server_key_iv: "03a504fc1dd2b0b2e899097f0124cfdd"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.425109 28192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/wal/instance:
uuid: "a2594ee63db746f59e98c890cc1e03d7"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "272369108bdd150428cc2c1ef5a85398"
server_key_iv: "03a504fc1dd2b0b2e899097f0124cfdd"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.426467 28192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:21.427390 28207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:21.427590 28192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:21.427695 28192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/wal
uuid: "a2594ee63db746f59e98c890cc1e03d7"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "272369108bdd150428cc2c1ef5a85398"
server_key_iv: "03a504fc1dd2b0b2e899097f0124cfdd"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.427754 28192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:21.440441 28192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:21.442631 28192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:21.442749 28192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:21.446719 28192 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.190:40575
I20260620 11:42:21.446777 28259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.190:40575 every 8 connection(s)
I20260620 11:42:21.447129 28192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/data/info.pb
I20260620 11:42:21.447568 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 28192
I20260620 11:42:21.447655 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-0/wal/instance
I20260620 11:42:21.447875 5610 external_mini_cluster.cc:1428] Setting key 0d09433aa1f73f2e02e60634df8279b2
I20260620 11:42:21.448307 28260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:21.450480 28260 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.122.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.189@KRBTEST.COM" created.
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:21.464039 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.451334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:40575 (local address 127.0.0.1:46390)
0620 11:42:21.451507 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.451552 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.452055 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.452369 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.452372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.452670 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.452829 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.452833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.453882 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.453884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.454656 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.454658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.455218 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.455219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.455283 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.455902 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.455911 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.459762 (+ 3851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.462356 (+ 2594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.462365 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.462367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.462591 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.462898 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.462899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.462899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.462988 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.463515 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.463521 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.463589 (+ 68us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.463700 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.463851 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260620 11:42:21.464147 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.451371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.5.122.190:40575)
0620 11:42:21.451584 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.451589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.451636 (+ 47us) server_negotiation.cc:407] Connection header received
0620 11:42:21.452237 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.452240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.452260 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.452364 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.452932 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.453767 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.454761 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.455094 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.455231 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.459917 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.459929 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.459929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.459939 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.462236 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.462692 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.462696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.462697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.462824 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.463057 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.463058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.463058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.463432 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.463531 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.463770 (+ 239us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.463945 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":189,"thread_start_us":144,"threads_started":1}
I20260620 11:42:21.464967 28260 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:21.465353 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.465222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60028)
0620 11:42:21.465274 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.465329 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260620 11:42:21.465495 28210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
I20260620 11:42:21.466004 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.465892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60036)
0620 11:42:21.465936 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.465981 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260620 11:42:21.466174 28260 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:39717: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab.
Entry for principal kudu/127.5.122.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab.
I20260620 11:42:21.477628 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:39717
--webserver_interface=127.5.122.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab
--principal=kudu/127.5.122.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:40575,127.5.122.189:39717,127.5.122.188:43785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:42:21.515401 28260 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } attempt: 1
I20260620 11:42:21.515940 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.515796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60042)
0620 11:42:21.515865 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.515910 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 11:42:21.516324 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.516219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60044)
0620 11:42:21.516270 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.516298 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 11:42:21.516408 28260 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:39717: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
W20260620 11:42:21.562812 28267 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 11:42:21.563118 28267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:21.563163 28267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:21.563211 28267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:21.564467 28267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:21.564563 28267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:21.564592 28267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:21.564612 28267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:21.564642 28267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:21.566285 28267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:40575,127.5.122.189:39717,127.5.122.188:43785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.189.keytab
--principal=kudu/127.5.122.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.189:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.122.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:21.566609 28267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:21.566859 28267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:21.572151 28273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:21.572187 28267 server_base.cc:1034] running on GCE node
W20260620 11:42:21.572149 28275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:21.572144 28272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:21.572685 28267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:21.573084 28267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:21.574258 28267 hybrid_clock.cc:648] HybridClock initialized: now 1781955741574233 us; error 37 us; skew 500 ppm
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:21.576912 28267 init.cc:377] Logged in from keytab as kudu/127.5.122.189@KRBTEST.COM (short username kudu)
I20260620 11:42:21.577452 28267 webserver.cc:460] Webserver started at http://127.5.122.189:37379/ using document root <none> and password file <none>
I20260620 11:42:21.577698 28267 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:21.577771 28267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:21.577927 28267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:21.579072 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data/instance:
uuid: "6e22d8650f434384a799a137a04ec183"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "2ea244620b45d0091ba7a0100ceb5211"
server_key_iv: "034d73522421a9b22c804afaef999458"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.579456 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/wal/instance:
uuid: "6e22d8650f434384a799a137a04ec183"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "2ea244620b45d0091ba7a0100ceb5211"
server_key_iv: "034d73522421a9b22c804afaef999458"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.580885 28267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260620 11:42:21.581699 28282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:21.581890 28267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:21.582020 28267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/wal
uuid: "6e22d8650f434384a799a137a04ec183"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "2ea244620b45d0091ba7a0100ceb5211"
server_key_iv: "034d73522421a9b22c804afaef999458"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.582115 28267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:21.584558 28260 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } attempt: 2
I20260620 11:42:21.584995 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.584868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60058)
0620 11:42:21.584928 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.584967 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 11:42:21.585424 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.585336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60062)
0620 11:42:21.585383 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.585409 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260620 11:42:21.585516 28260 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.189:39717: Network error: Client connection negotiation failed: client connection to 127.5.122.189:39717: connect: Connection refused (error 111)
I20260620 11:42:21.595395 28267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:21.597522 28267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:21.597715 28267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:21.602355 28267 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.189:39717
I20260620 11:42:21.602391 28334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.189:39717 every 8 connection(s)
I20260620 11:42:21.602852 28267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/data/info.pb
I20260620 11:42:21.603076 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 28267
I20260620 11:42:21.603230 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-1/wal/instance
I20260620 11:42:21.603430 5610 external_mini_cluster.cc:1428] Setting key 04886e48216ffa23318d8a3a26c1783b
I20260620 11:42:21.604091 28335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:21.605818 28335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.122.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.122.188@KRBTEST.COM" created.
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:21.619618 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.606896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.5.122.190:40575)
0620 11:42:21.606979 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.606982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.607309 (+ 327us) server_negotiation.cc:407] Connection header received
0620 11:42:21.607875 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.607876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.607892 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.607969 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.608638 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.609126 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.610251 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.610464 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.610543 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.615664 (+ 5121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.615667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.615667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.615702 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.617543 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.618116 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.618118 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.618119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.618193 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.618649 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.618650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.618651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.618828 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.618907 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.619372 (+ 465us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.619480 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 11:42:21.619719 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.606862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:40575 (local address 127.0.0.1:46392)
0620 11:42:21.607156 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.607202 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.607757 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.608044 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.608048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.608320 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.608536 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.608540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.609236 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.609238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.610160 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.610162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.610580 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.610582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.610740 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.611360 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.611374 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.615485 (+ 4111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.617664 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.617677 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.617680 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.617955 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.618300 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.618301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.618301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.618551 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.618955 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.618965 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.619188 (+ 223us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.619295 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.619467 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":193,"threads_started":1}
I20260620 11:42:21.620496 28335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:42:21.629168 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.620837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60078)
0620 11:42:21.620899 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.620911 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.620959 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.621288 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.621289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.621460 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.621543 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.621545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.622609 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.622610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.623188 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.623190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.623756 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.623757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.623832 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.624364 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.624376 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.626222 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.627987 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.627988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.627989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.628175 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.628419 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.628420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.628421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.628479 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.628824 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.628825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.628866 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.628928 (+ 62us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.629047 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260620 11:42:21.629204 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.620917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.5.122.189:39717)
0620 11:42:21.621124 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.621129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.621149 (+ 20us) server_negotiation.cc:407] Connection header received
0620 11:42:21.621168 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.621170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.621182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.621270 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.621636 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.622499 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.623284 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.623668 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.623761 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.626340 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.626354 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.626354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.626361 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.627875 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.628282 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.628286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.628287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.628348 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.628567 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.628568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.628568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.628749 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.628849 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.628969 (+ 120us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.629102 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
I20260620 11:42:21.630017 28335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:21.630625 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.630432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33834)
0620 11:42:21.630532 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.630597 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260620 11:42:21.630784 28284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
I20260620 11:42:21.631212 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.631106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33850)
0620 11:42:21.631162 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.631189 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260620 11:42:21.631325 28335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:43785: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.122.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab.
Entry for principal kudu/127.5.122.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab.
I20260620 11:42:21.634195 5610 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIe33o/build/release/bin/kudu
/tmp/dist-test-taskJIe33o/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:43785
--webserver_interface=127.5.122.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab
--principal=kudu/127.5.122.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.122.148:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.122.190:40575,127.5.122.189:39717,127.5.122.188:43785
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 11:42:21.676702 28260 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } attempt: 3
I20260620 11:42:21.680572 28335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } attempt: 1
I20260620 11:42:21.681005 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.680863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33860)
0620 11:42:21.680933 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.680964 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:42:21.681413 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.681297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33870)
0620 11:42:21.681342 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.681367 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 11:42:21.681576 28335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:43785: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
I20260620 11:42:21.684640 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.677207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60094)
0620 11:42:21.677283 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.677298 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.677370 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.677580 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.677581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.677758 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.677823 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.677824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.678604 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.678610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.679188 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.679190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.679610 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.679611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.679693 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.680200 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.680204 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.681741 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.683525 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.683526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.683527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.683773 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.683979 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.683980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.683980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.684044 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.684272 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.684272 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.684329 (+ 57us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.684427 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.684531 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260620 11:42:21.684613 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.677328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.5.122.189:39717)
0620 11:42:21.677403 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.677406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.677417 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:42:21.677447 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.677448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.677467 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.677568 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.677916 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.678490 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.679283 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.679517 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.679608 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.681835 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.681836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.681836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.681842 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.683431 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.683853 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.683854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.683854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.683906 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.684103 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.684103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.684103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.684210 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.684257 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.684443 (+ 186us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.684555 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260620 11:42:21.685133 28260 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } has no permanent_uuid. Determining permanent_uuid...
I20260620 11:42:21.685520 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.685405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33874)
0620 11:42:21.685457 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.685494 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260620 11:42:21.685901 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.685808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33888)
0620 11:42:21.685854 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.685883 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260620 11:42:21.686023 28260 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:43785: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
I20260620 11:42:21.717227 28260 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } attempt: 1
I20260620 11:42:21.717832 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.717677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33896)
0620 11:42:21.717746 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.717795 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260620 11:42:21.718283 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.718188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33904)
0620 11:42:21.718222 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.718254 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260620 11:42:21.718451 28260 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:43785: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
W20260620 11:42:21.723052 28342 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 11:42:21.723300 28342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 11:42:21.723318 28342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 11:42:21.723341 28342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 11:42:21.724606 28342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 11:42:21.724655 28342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 11:42:21.724670 28342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 11:42:21.724684 28342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 11:42:21.724695 28342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 11:42:21.726398 28342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.122.148:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.122.190:40575,127.5.122.189:39717,127.5.122.188:43785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/krb5kdc/kudu_127.5.122.188.keytab
--principal=kudu/127.5.122.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.122.188:43785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.122.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 20 Jun 2026 10:32:37 UTC on ccycloud.quasar-bgyofv.root.comops.site
build id 4457173
I20260620 11:42:21.726701 28342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 11:42:21.726958 28342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 11:42:21.732194 28348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:21.732153 28347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 11:42:21.732153 28350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 11:42:21.732508 28342 server_base.cc:1034] running on GCE node
I20260620 11:42:21.732724 28342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 11:42:21.732983 28342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 11:42:21.734135 28342 hybrid_clock.cc:648] HybridClock initialized: now 1781955741734110 us; error 37 us; skew 500 ppm
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 11:42:21.736722 28342 init.cc:377] Logged in from keytab as kudu/127.5.122.188@KRBTEST.COM (short username kudu)
I20260620 11:42:21.737213 28342 webserver.cc:460] Webserver started at http://127.5.122.188:42169/ using document root <none> and password file <none>
I20260620 11:42:21.737416 28342 fs_manager.cc:359] Metadata directory not provided
I20260620 11:42:21.737458 28342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 11:42:21.737555 28342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 11:42:21.738705 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data/instance:
uuid: "fdf8364814134418a1859bf770371cce"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "968673cd679c76025cc725af8a69ebaa"
server_key_iv: "48717657afbbeb0057f1040262edd4c4"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.739109 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/wal/instance:
uuid: "fdf8364814134418a1859bf770371cce"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "968673cd679c76025cc725af8a69ebaa"
server_key_iv: "48717657afbbeb0057f1040262edd4c4"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.740477 28342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260620 11:42:21.741288 28357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 11:42:21.741534 28342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 11:42:21.741614 28342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/wal
uuid: "fdf8364814134418a1859bf770371cce"
format_stamp: "Formatted at 2026-06-20 11:42:21 on dist-test-slave-szdr"
server_key: "968673cd679c76025cc725af8a69ebaa"
server_key_iv: "48717657afbbeb0057f1040262edd4c4"
server_key_version: "encryptionkey@0"
I20260620 11:42:21.741701 28342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 11:42:21.749730 28335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } attempt: 2
I20260620 11:42:21.750281 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.750145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33916)
0620 11:42:21.750208 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.750251 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 11:42:21.750708 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.750616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33932)
0620 11:42:21.750662 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.750690 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260620 11:42:21.750854 28335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.122.188:43785: Network error: Client connection negotiation failed: client connection to 127.5.122.188:43785: connect: Connection refused (error 111)
I20260620 11:42:21.760193 28342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 11:42:21.762360 28342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 11:42:21.762506 28342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 11:42:21.766642 28342 rpc_server.cc:307] RPC server started. Bound to: 127.5.122.188:43785
I20260620 11:42:21.766716 28409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.122.188:43785 every 8 connection(s)
I20260620 11:42:21.767057 28342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/data/info.pb
I20260620 11:42:21.768121 28410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 11:42:21.769484 5610 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIe33o/build/release/bin/kudu as pid 28342
I20260620 11:42:21.769567 5610 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIe33o/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781955466149030-5610-0/minicluster-data/master-2/wal/instance
I20260620 11:42:21.769816 5610 external_mini_cluster.cc:1428] Setting key bcac59e74db65c2876ed0f85a043c180
I20260620 11:42:21.769942 28410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:42:21.779554 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.770826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.5.122.190:40575)
0620 11:42:21.770881 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.770883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.771026 (+ 143us) server_negotiation.cc:407] Connection header received
0620 11:42:21.771177 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.771178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.771189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.771251 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.771866 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.772396 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.773455 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.773677 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.773773 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.776272 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.776274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.776274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.776289 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.778104 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.778549 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.778550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.778551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.778615 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.778872 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.778873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.778874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.779015 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.779092 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.779356 (+ 264us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.779439 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260620 11:42:21.779781 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.770339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.5.122.188:43785)
0620 11:42:21.770492 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.770500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.770524 (+ 24us) server_negotiation.cc:407] Connection header received
0620 11:42:21.771103 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.771111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.771138 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.771246 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.771890 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.772663 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.773522 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.773831 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.773922 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.776562 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.776570 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.776570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.776583 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.778338 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.778715 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.778718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.778718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.778796 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.779026 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.779027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.779027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.779228 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.779380 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.779464 (+ 84us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.779577 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260620 11:42:21.779780 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.770745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.190:40575 (local address 127.0.0.1:46402)
0620 11:42:21.770891 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.770942 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.771090 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.771271 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.771274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.771574 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.771750 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.771755 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.772506 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.772507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.773363 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.773365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.773806 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.773808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.773918 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.774655 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.774674 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.776125 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.778200 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.778206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.778209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.778449 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.778688 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.778689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.778689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.778798 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.779099 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.779106 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.779227 (+ 121us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.779332 (+ 105us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.779506 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260620 11:42:21.780452 28410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.190@KRBTEST.COM
I20260620 11:42:21.787914 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.780809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.5.122.189:39717)
0620 11:42:21.780864 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.780867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.780877 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:42:21.780903 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.780904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.780914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.780965 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.781248 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.781764 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.782557 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.782801 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.782884 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.784823 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.784825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.784825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.784833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.786646 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.787080 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.787081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.787081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.787142 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.787367 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.787368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.787369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.787495 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.787572 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.787742 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.787813 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260620 11:42:21.787914 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.780736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.189:39717 (local address 127.0.0.1:60098)
0620 11:42:21.780776 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.780787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.780841 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.780984 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.780985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.781128 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.781165 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.781167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.781850 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.781851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.782454 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.782456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.782880 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.782881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.782953 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.783475 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.783480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.784731 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.786753 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.786755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.786755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.786938 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.787225 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.787226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.787226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.787293 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.787578 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.787579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.787622 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.787715 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.787814 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260620 11:42:21.788046 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.780730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.5.122.190:40575)
0620 11:42:21.780771 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.780773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.780783 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:42:21.780919 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.780920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.780931 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.780987 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.781434 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.781870 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.782548 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.782735 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.782820 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.785351 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.785353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.785353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.785360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.786859 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.787223 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.787224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.787225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.787278 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.787476 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.787477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.787477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.787590 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.787750 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.787787 (+ 37us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.787854 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:42:21.788282 28410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.188@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.122.189@KRBTEST.COM
I20260620 11:42:21.793601 28260 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } attempt: 2
I20260620 11:42:21.797171 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.788525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33958)
0620 11:42:21.788573 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.788582 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.788616 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.788853 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.788854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.789015 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.789052 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.789054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.789789 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.789790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.790650 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.790653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.791053 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.791054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.791093 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.791528 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.791534 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.793132 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.796089 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.796090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.796090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.796306 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.796533 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.796534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.796534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.796589 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.796879 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.796880 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.796915 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.796984 (+ 69us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.797073 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260620 11:42:21.797258 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.788663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.5.122.189:39717)
0620 11:42:21.788696 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.788698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.788730 (+ 32us) server_negotiation.cc:407] Connection header received
0620 11:42:21.788790 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.788790 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.788800 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.788851 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.789408 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.790065 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.790980 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.791226 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.791302 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.794192 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.794193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.794193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.794200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.795994 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.796405 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.796406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.796406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.796466 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.796717 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.796718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.796719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.796846 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.797017 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.797090 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.797156 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260620 11:42:21.797436 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.788705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.5.122.188:43785)
0620 11:42:21.788741 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.788743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.788749 (+ 6us) server_negotiation.cc:407] Connection header received
0620 11:42:21.788766 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.788766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.788776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.788833 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.789132 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.789710 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.790744 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.790969 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.791045 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.793218 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.793226 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.793227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.793235 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.795181 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.796386 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.796387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.796388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.796443 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.796668 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.796669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.796669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.796795 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.796857 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.797277 (+ 420us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.797345 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:42:21.799182 5610 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 11:42:21.799319 28410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: Bootstrap starting.
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.190@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:42:21.800107 28410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:21.800480 28410 log.cc:824] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:21.800354 28234 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a2594ee63db746f59e98c890cc1e03d7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46414
I20260620 11:42:21.801630 28410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: No bootstrap required, opened a new log
I20260620 11:42:21.802786 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.793851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33968)
0620 11:42:21.793904 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.793915 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.793952 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.794290 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.794291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.794466 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.794510 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.794513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.795334 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.795335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.796176 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.796179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.797729 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.797730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.797801 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.798471 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.798475 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.799938 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.801724 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.801725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.801725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.801896 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.802121 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.802122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.802122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.802180 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.802455 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.802456 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.802501 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.802585 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.802685 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260620 11:42:21.802817 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.793921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33968 (local address 127.5.122.188:43785)
0620 11:42:21.794118 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.794120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.794127 (+ 7us) server_negotiation.cc:407] Connection header received
0620 11:42:21.794144 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.794145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.794156 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.794253 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.794608 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.795265 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.796265 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.797127 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.798330 (+ 1203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.800045 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.800046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.800046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.800053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.801627 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.801988 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.801990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.801990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.802045 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.802249 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.802250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.802250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.802382 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.802460 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.802622 (+ 162us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.802693 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":139,"threads_started":1}
I20260620 11:42:21.803684 28410 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.803843 28410 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf8364814134418a1859bf770371cce, State: Initialized, Role: FOLLOWER
I20260620 11:42:21.803882 28260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7: Bootstrap starting.
I20260620 11:42:21.803933 28410 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.804293 28410 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:21.804244 28417 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.804401 28260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:21.804530 28417 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:21.804613 28260 log.cc:824] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:21.805404 28260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7: No bootstrap required, opened a new log
W20260620 11:42:21.806469 28429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:42:21.806533 28429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:42:21.806807 28260 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.806950 28260 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2594ee63db746f59e98c890cc1e03d7, State: Initialized, Role: FOLLOWER
I20260620 11:42:21.807032 28260 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.807377 28234 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 11:42:21.807312 28431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.807446 28234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:21.807540 28431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:21.808266 28234 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.808494 28234 leader_election.cc:288] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [CANDIDATE]: Term 1 election: Requested vote from peers 6e22d8650f434384a799a137a04ec183 (127.5.122.189:39717), fdf8364814134418a1859bf770371cce (127.5.122.188:43785)
I20260620 11:42:21.806787 28342 master_runner.cc:186] Error getting master registration for 127.5.122.190:40575: OK, instance_id { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" instance_seqno: 1781955741445523 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 11:42:21.808917 28342 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 11:42:21.809576 28384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a2594ee63db746f59e98c890cc1e03d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fdf8364814134418a1859bf770371cce"
I20260620 11:42:21.809685 28384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:21.810542 28384 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2594ee63db746f59e98c890cc1e03d7 in term 1.
I20260620 11:42:21.810827 28209 leader_election.cc:302] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2594ee63db746f59e98c890cc1e03d7, fdf8364814134418a1859bf770371cce; no voters:
I20260620 11:42:21.810959 28431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 11:42:21.811089 28431 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [term 1 LEADER]: Becoming Leader. State: Replica: a2594ee63db746f59e98c890cc1e03d7, State: Running, Role: LEADER
I20260620 11:42:21.811180 28431 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.811583 28431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2594ee63db746f59e98c890cc1e03d7. Latest consensus state: current_term: 1 leader_uuid: "a2594ee63db746f59e98c890cc1e03d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.811672 28431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:21.807312 28260 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 11:42:21.817770 28452 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 11:42:21.817835 28452 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 11:42:21.814750 28192 master_runner.cc:186] Error getting master registration for 127.5.122.189:39717: OK, instance_id { permanent_uuid: "6e22d8650f434384a799a137a04ec183" instance_seqno: 1781955741600752 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 11:42:21.818305 28192 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 11:42:21.818871 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.810046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.5.122.188:43785)
0620 11:42:21.810085 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.810086 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.810278 (+ 192us) server_negotiation.cc:407] Connection header received
0620 11:42:21.810340 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.810341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.810353 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.810408 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.811177 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.811914 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.812897 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.813224 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.813263 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.815807 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.815810 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.815810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.815819 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.817466 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.817848 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.817850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.817850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.817911 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.818196 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.818197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.818197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.818330 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.818424 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.818674 (+ 250us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.818763 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:42:21.818979 28441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 11:42:21.819145 28441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 11:42:21.820847 28384 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [term 1 FOLLOWER]: Refusing update from remote peer a2594ee63db746f59e98c890cc1e03d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:42:21.822095 28431 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:42:21.823357 28417 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2594ee63db746f59e98c890cc1e03d7. Latest consensus state: current_term: 1 leader_uuid: "a2594ee63db746f59e98c890cc1e03d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.823441 28417 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:21.824071 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.810685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.5.122.189:39717)
0620 11:42:21.810743 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.810745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.810755 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:42:21.810781 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.810782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.810794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.810854 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.811826 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.812565 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.813547 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.814342 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.814391 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.820158 (+ 5767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.820159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.820160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.820168 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.822011 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.822661 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.822662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.822663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.822720 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.823111 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.823112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.823113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.823324 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.823492 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.823794 (+ 302us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.823934 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260620 11:42:21.830848 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.811847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.5.122.190:40575)
0620 11:42:21.811881 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.811882 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.811890 (+ 8us) server_negotiation.cc:407] Connection header received
0620 11:42:21.811908 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.811909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.811917 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.811986 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.814795 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.815559 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.816568 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.816866 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.816908 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.820001 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.820002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.820003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.820009 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.821664 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.823840 (+ 2176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.823841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.823841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.823900 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.830213 (+ 6313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.830215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.830215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.830375 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.830481 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.830659 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.830738 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260620 11:42:21.831409 28431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2594ee63db746f59e98c890cc1e03d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.831507 28431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:21.831717 28454 mvcc.cc:205] Tried to move back new op lower bound from 7298890718495944704 to 7298890718459371520. Current Snapshot: MvccSnapshot[applied={T|T < 7298890718495944704}]
I20260620 11:42:21.831974 28417 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2594ee63db746f59e98c890cc1e03d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.832064 28417 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:21.832150 28441 catalog_manager.cc:1347] Generated new cluster ID: 46dc5a69af2f4c1f99398d53d1ed6e52
I20260620 11:42:21.832190 28441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 11:42:21.832237 28455 mvcc.cc:205] Tried to move back new op lower bound from 7298890718495944704 to 7298890718459371520. Current Snapshot: MvccSnapshot[applied={T|T < 7298890718495944704}]
I20260620 11:42:21.842008 28335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } attempt: 3
I20260620 11:42:21.843223 28441 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 11:42:21.844389 28441 catalog_manager.cc:1504] Loading token signing keys...
Jun 20 11:42:21 dist-test-slave-szdr krb5kdc[28179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781955741, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.122.189@KRBTEST.COM for kudu/127.5.122.188@KRBTEST.COM
I20260620 11:42:21.850723 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.842278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.122.188:43785 (local address 127.0.0.1:33994)
0620 11:42:21.842354 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0620 11:42:21.842365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0620 11:42:21.842403 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 11:42:21.842530 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 11:42:21.842531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 11:42:21.842718 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0620 11:42:21.842766 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.842768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.843692 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.843693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.844561 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 11:42:21.844565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.845100 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.845100 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 11:42:21.845158 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.845722 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 11:42:21.845729 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0620 11:42:21.847327 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 11:42:21.849414 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.849415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.849415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.849654 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.849905 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 11:42:21.849906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 11:42:21.849906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0620 11:42:21.850027 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 11:42:21.850364 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 11:42:21.850365 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 11:42:21.850430 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 11:42:21.850493 (+ 63us) client_negotiation.cc:240] Negotiation successful
0620 11:42:21.850618 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 11:42:21.852095 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.842374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33994 (local address 127.5.122.188:43785)
0620 11:42:21.842411 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.842413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.842424 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:42:21.842457 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.842458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.842468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.842521 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.842850 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.843601 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.844676 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.844989 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.845046 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.847424 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 11:42:21.847425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 11:42:21.847426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 11:42:21.847433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0620 11:42:21.849278 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.849752 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.849753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.849754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.849827 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 11:42:21.850115 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 11:42:21.850116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 11:42:21.850117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 11:42:21.850267 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 11:42:21.850325 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.851894 (+ 1569us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.851997 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260620 11:42:21.853668 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183: Bootstrap starting.
I20260620 11:42:21.854488 28335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183: Neither blocks nor log segments found. Creating new log.
I20260620 11:42:21.854805 28335 log.cc:824] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183: Log is configured to *not* fsync() on all Append() calls
I20260620 11:42:21.855118 28441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7: Generated new TSK 0
I20260620 11:42:21.855468 28441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 11:42:21.855986 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183: No bootstrap required, opened a new log
I20260620 11:42:21.860234 28335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.860414 28335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e22d8650f434384a799a137a04ec183, State: Initialized, Role: FOLLOWER
I20260620 11:42:21.860507 28335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } }
I20260620 11:42:21.860909 28309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a2594ee63db746f59e98c890cc1e03d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6e22d8650f434384a799a137a04ec183"
I20260620 11:42:21.861071 28309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [term 0 FOLLOWER]: Advancing to term 1
I20260620 11:42:21.862022 28309 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2594ee63db746f59e98c890cc1e03d7 in term 1.
I20260620 11:42:21.860889 28335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 11:42:21.862186 28308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [term 1 FOLLOWER]: Refusing update from remote peer a2594ee63db746f59e98c890cc1e03d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 11:42:21.862653 28439 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 11:42:21.860859 28461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.863679 28461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:21.865144 28473 mvcc.cc:205] Tried to move back new op lower bound from 7298890718630588416 to 7298890718459371520. Current Snapshot: MvccSnapshot[applied={T|T < 7298890718495944704}]
I20260620 11:42:21.868101 28479 catalog_manager.cc:1259] Loaded cluster ID: 46dc5a69af2f4c1f99398d53d1ed6e52
I20260620 11:42:21.868160 28479 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183: loading cluster ID for follower catalog manager: success
I20260620 11:42:21.869344 28479 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183: acquiring CA information for follower catalog manager: success
I20260620 11:42:21.869692 28479 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 11:42:21.867429 28461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2594ee63db746f59e98c890cc1e03d7. Latest consensus state: current_term: 1 leader_uuid: "a2594ee63db746f59e98c890cc1e03d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.870208 28461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 11:42:21.870296 28461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2594ee63db746f59e98c890cc1e03d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:21.870362 28461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e22d8650f434384a799a137a04ec183 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 11:42:21.882759 28340 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.880993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.5.122.189:39717)
0620 11:42:21.881043 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.881046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.882021 (+ 975us) server_negotiation.cc:407] Connection header received
0620 11:42:21.882136 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.882137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.882149 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.882181 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.882676 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60122: BlockingRecv error: recv got EOF from 127.0.0.1:60122 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260620 11:42:21.883692 28264 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.881411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.5.122.190:40575)
0620 11:42:21.881455 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.881457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.883050 (+ 1593us) server_negotiation.cc:407] Connection header received
0620 11:42:21.883253 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.883254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.883267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.883297 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.883622 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46438: BlockingRecv error: recv got EOF from 127.0.0.1:46438 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 11:42:21.884389 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.881707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.5.122.188:43785)
0620 11:42:21.881746 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.881748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.883147 (+ 1399us) server_negotiation.cc:407] Connection header received
0620 11:42:21.883912 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.883913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.883938 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.883969 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.884324 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34006: BlockingRecv error: recv got EOF from 127.0.0.1:34006 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:42:21.890535 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.886306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.5.122.189:39717)
0620 11:42:21.886348 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.886350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.886632 (+ 282us) server_negotiation.cc:407] Connection header received
0620 11:42:21.886756 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.886757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.886813 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.886864 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:21.887173 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.888000 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.889404 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.889744 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.889790 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.890085 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:21.890244 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:21.890300 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.890417 (+ 117us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.890440 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 11:42:21.894651 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.893546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.5.122.188:43785)
0620 11:42:21.893606 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.893608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.893932 (+ 324us) server_negotiation.cc:407] Connection header received
0620 11:42:21.894092 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.894094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.894119 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.894152 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.894583 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34010: BlockingRecv error: recv got EOF from 127.0.0.1:34010 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260620 11:42:21.901232 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.897221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.5.122.189:39717)
0620 11:42:21.897266 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.897269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.897581 (+ 312us) server_negotiation.cc:407] Connection header received
0620 11:42:21.897690 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.897692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.897704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.897733 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:21.897920 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.898816 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.900331 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.900659 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.900706 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.900935 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:21.900977 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:21.901004 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.901110 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.901138 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260620 11:42:21.905318 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.904194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.5.122.188:43785)
0620 11:42:21.904242 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.904244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.904652 (+ 408us) server_negotiation.cc:407] Connection header received
0620 11:42:21.904785 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.904787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.904801 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.904832 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.905247 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34022: BlockingRecv error: recv got EOF from 127.0.0.1:34022 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:42:21.912660 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.908942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.5.122.189:39717)
0620 11:42:21.908994 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.908996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.909336 (+ 340us) server_negotiation.cc:407] Connection header received
0620 11:42:21.909443 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.909444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.909457 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.909500 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:21.909701 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.910565 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.911789 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.912097 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.912142 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.912356 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:21.912395 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:21.912422 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.912547 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.912571 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260620 11:42:21.916327 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.915355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.5.122.188:43785)
0620 11:42:21.915400 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.915402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.915705 (+ 303us) server_negotiation.cc:407] Connection header received
0620 11:42:21.915818 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.915819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.915833 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.915864 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.916267 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34036: BlockingRecv error: recv got EOF from 127.0.0.1:34036 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:42:21.925505 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.921831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.5.122.189:39717)
0620 11:42:21.921881 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.921883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.922238 (+ 355us) server_negotiation.cc:407] Connection header received
0620 11:42:21.922348 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.922349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.922361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.922391 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:21.922598 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.923370 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.924632 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.924948 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.924993 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.925210 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:21.925247 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:21.925274 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.925379 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.925403 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 11:42:21.929138 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.928165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34050 (local address 127.5.122.188:43785)
0620 11:42:21.928211 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.928213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.928518 (+ 305us) server_negotiation.cc:407] Connection header received
0620 11:42:21.928640 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.928641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.928654 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.928685 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.929073 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34050: BlockingRecv error: recv got EOF from 127.0.0.1:34050 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260620 11:42:21.942358 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.938717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60166 (local address 127.5.122.189:39717)
0620 11:42:21.938765 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.938767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.939130 (+ 363us) server_negotiation.cc:407] Connection header received
0620 11:42:21.939246 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.939247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.939259 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.939305 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:21.939582 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.940337 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.941475 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.941780 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.941823 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.942072 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:21.942111 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:21.942139 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.942244 (+ 105us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.942267 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260620 11:42:21.946105 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.945034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.5.122.188:43785)
0620 11:42:21.945077 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.945079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.945362 (+ 283us) server_negotiation.cc:407] Connection header received
0620 11:42:21.945465 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.945465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.945478 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.945519 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.946040 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34058: BlockingRecv error: recv got EOF from 127.0.0.1:34058 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 11:42:21.967216 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:21.963560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.5.122.189:39717)
0620 11:42:21.963607 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.963609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.963956 (+ 347us) server_negotiation.cc:407] Connection header received
0620 11:42:21.964074 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.964075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.964087 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.964123 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:21.964324 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.965082 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.966341 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:21.966639 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:21.966692 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:21.966931 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:21.966968 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:21.966997 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:21.967105 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 11:42:21.967128 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260620 11:42:21.971067 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:21.970066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.5.122.188:43785)
0620 11:42:21.970109 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:21.970112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:21.970381 (+ 269us) server_negotiation.cc:407] Connection header received
0620 11:42:21.970533 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:21.970535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:21.970549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:21.970583 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:21.970999 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34066: BlockingRecv error: recv got EOF from 127.0.0.1:34066 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260620 11:42:22.008244 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:22.004578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.5.122.189:39717)
0620 11:42:22.004625 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.004626 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.005019 (+ 393us) server_negotiation.cc:407] Connection header received
0620 11:42:22.005128 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.005129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.005141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.005171 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:22.005366 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.006230 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.007414 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.007717 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.007760 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:22.007965 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:22.008002 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:22.008028 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:22.008126 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 11:42:22.008148 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260620 11:42:22.012076 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:22.011107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.5.122.188:43785)
0620 11:42:22.011152 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.011154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.011373 (+ 219us) server_negotiation.cc:407] Connection header received
0620 11:42:22.011470 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.011472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.011495 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.011526 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:22.011919 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34082: BlockingRecv error: recv got EOF from 127.0.0.1:34082 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:42:22.081074 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:22.077815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60192 (local address 127.5.122.189:39717)
0620 11:42:22.077888 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.077890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.078082 (+ 192us) server_negotiation.cc:407] Connection header received
0620 11:42:22.078176 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.078177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.078192 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.078269 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:22.078410 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.079192 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.080457 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.080650 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.080721 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:22.080842 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:22.080869 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:22.080898 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:22.080940 (+ 42us) server_negotiation.cc:299] Negotiation successful
0620 11:42:22.080956 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260620 11:42:22.083563 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:22.082993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34090 (local address 127.5.122.188:43785)
0620 11:42:22.083036 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.083038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.083108 (+ 70us) server_negotiation.cc:407] Connection header received
0620 11:42:22.083149 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.083150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.083161 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.083209 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:22.083473 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34090: BlockingRecv error: recv got EOF from 127.0.0.1:34090 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260620 11:42:22.215350 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:22.212733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.5.122.189:39717)
0620 11:42:22.212789 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.212791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.212865 (+ 74us) server_negotiation.cc:407] Connection header received
0620 11:42:22.212909 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.212910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.212920 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.212991 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:22.213135 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.213605 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.214488 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.214689 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.214753 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:22.214894 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:22.214923 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:22.214961 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:22.215266 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 11:42:22.215286 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260620 11:42:22.217957 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:22.217236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.5.122.188:43785)
0620 11:42:22.217317 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.217319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.217389 (+ 70us) server_negotiation.cc:407] Connection header received
0620 11:42:22.217480 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.217480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.217492 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.217564 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:22.217884 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34096: BlockingRecv error: recv got EOF from 127.0.0.1:34096 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 11:42:22.477694 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:22.475442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60204 (local address 127.5.122.189:39717)
0620 11:42:22.475517 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.475524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.475534 (+ 10us) server_negotiation.cc:407] Connection header received
0620 11:42:22.475560 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.475561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.475604 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.475657 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:22.475808 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.476295 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.477062 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.477247 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.477304 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:22.477464 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:22.477493 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:22.477535 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:22.477612 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 11:42:22.477629 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260620 11:42:22.480186 28415 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 11:42:22.479576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.5.122.188:43785)
0620 11:42:22.479645 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.479648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.479659 (+ 11us) server_negotiation.cc:407] Connection header received
0620 11:42:22.479690 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.479690 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.479702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.479780 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 11:42:22.480128 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34098: BlockingRecv error: recv got EOF from 127.0.0.1:34098 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260620 11:42:22.482623 28579 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2594ee63db746f59e98c890cc1e03d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2594ee63db746f59e98c890cc1e03d7" member_type: VOTER last_known_addr { host: "127.5.122.190" port: 40575 } } peers { permanent_uuid: "6e22d8650f434384a799a137a04ec183" member_type: VOTER last_known_addr { host: "127.5.122.189" port: 39717 } } peers { permanent_uuid: "fdf8364814134418a1859bf770371cce" member_type: VOTER last_known_addr { host: "127.5.122.188" port: 43785 } } }
I20260620 11:42:22.482801 28579 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2594ee63db746f59e98c890cc1e03d7 [sys.catalog]: This master's current role is: LEADER
I20260620 11:42:22.806978 28429 catalog_manager.cc:1259] Loaded cluster ID: 46dc5a69af2f4c1f99398d53d1ed6e52
I20260620 11:42:22.807102 28429 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: loading cluster ID for follower catalog manager: success
I20260620 11:42:22.807924 28429 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: acquiring CA information for follower catalog manager: success
I20260620 11:42:22.808259 28429 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fdf8364814134418a1859bf770371cce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 11:42:22.996454 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:22.993616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.5.122.189:39717)
0620 11:42:22.993765 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.993767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.993862 (+ 95us) server_negotiation.cc:407] Connection header received
0620 11:42:22.994049 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.994051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.994063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.994146 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:22.994300 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.994822 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.995650 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.995866 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:22.995969 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:22.996153 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:22.996194 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:22.996257 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:22.996319 (+ 62us) server_negotiation.cc:299] Negotiation successful
0620 11:42:22.996343 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260620 11:42:23.001060 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:22.998358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.5.122.188:43785)
0620 11:42:22.998502 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:22.998504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:22.998560 (+ 56us) server_negotiation.cc:407] Connection header received
0620 11:42:22.998618 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:22.998619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:22.998669 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:22.998742 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:22.998870 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:22.999369 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.000148 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:23.000343 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.000398 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:23.000625 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:23.000763 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:23.000860 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:23.000904 (+ 44us) server_negotiation.cc:299] Negotiation successful
0620 11:42:23.000927 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260620 11:42:23.006717 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:23.003310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34114 (local address 127.5.122.188:43785)
0620 11:42:23.003378 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:23.003380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:23.003436 (+ 56us) server_negotiation.cc:407] Connection header received
0620 11:42:23.003505 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:23.003506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:23.003517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:23.003573 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:23.003757 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:23.004501 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.005810 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:23.006157 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.006223 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:23.006409 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:23.006466 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:23.006543 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:23.006607 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 11:42:23.006630 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260620 11:42:23.006717 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:23.003126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.5.122.189:39717)
0620 11:42:23.003174 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:23.003176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:23.003343 (+ 167us) server_negotiation.cc:407] Connection header received
0620 11:42:23.003382 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:23.003383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:23.003399 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:23.003458 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:23.003655 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:23.004478 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.005805 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:23.006123 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.006174 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:23.006391 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:23.006435 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:23.006466 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:23.006574 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 11:42:23.006605 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260620 11:42:23.006891 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 11:42:23.003192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.5.122.190:40575)
0620 11:42:23.003298 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0620 11:42:23.003300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0620 11:42:23.003445 (+ 145us) server_negotiation.cc:407] Connection header received
0620 11:42:23.003564 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 11:42:23.003565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 11:42:23.003624 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 11:42:23.003728 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 11:42:23.003837 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:23.004595 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.005732 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 11:42:23.006069 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 11:42:23.006163 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 11:42:23.006478 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 11:42:23.006623 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 11:42:23.006685 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0620 11:42:23.006770 (+ 85us) server_negotiation.cc:299] Negotiation successful
0620 11:42:23.006802 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
I20260620 11:42:23.009526 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 28192
I20260620 11:42:23.015800 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 28267
I20260620 11:42:23.021884 5610 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIe33o/build/release/bin/kudu with pid 28342
2026-06-20T11:42:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2883 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5740 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (276881 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST